On Thu, Jun 28, 2012 at 8:26 PM, Chetan Jhurani <[email protected]>wrote:

...

Look. It does not do that. Ever. In any universe. I don't know what you think you are doing,****

...

but it is definitely not what you wrote. Look at the code. It can't do that.****

** **

I don’t think you understand what your code is doing in a new****

environment of the same universe.

Yes, you are correct. I do not understand this code. I was wrong to tell you to use it. Don't use it. Matt

It works fine on Linux with the steps you had given.****

But on the cygwin side, no dependency files like****

./lib/libpetsc-obj/adaptbasic.d are created. I found this after****

running the python build on Linux (I didn’t know before this****

where the dependencies were stored).****

** **

Here is some debug info in the python code:****

** **

In the buildLibraries function on the Linux box (when building in the second step)****

** **

self.sourceDatabase = DirectedGraph with 1172 vertices and 21894 edges* ***

** **

However, on the cygwin side (again, when building the second time)****

** **

self.sourceDatabase = <builder.NullSourceDatabase object at 0xffd833ac>* ***

** **

So of course it is totalRebuild = true on cygwin.****

** **

I don’t mind extra debugging on cygwin/Windows before reporting issues****

since I know the petsc project has many other priorities. But a few useful ****

debugging pointers will be serve both of us better.****

** **

Chetan****

** **

** **

** **

** **

*From:* [email protected] [mailto: [email protected]] *On Behalf Of *Matthew Knepley *Sent:* Thursday, June 28, 2012 6:00 PM

*To:* For users of the development version of PETSc *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin****

** **

On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <[email protected]> wrote:****

*> > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py

...

...

(The same happens (existing .o are deleted) if ./config/builder2.py build is used)*****

*> *This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options, *> *so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it.****

*> *I said to use builder2.py, which does proper dependency tracking.****

****

That is what I used and I mentioned it above. I’ve tried both separately.* ***

****

Both give the same result that libpetsc.lib is not created. And****

whether I use builder.py or builder2.py, the existing objects****

files are deleted in the second build. I did not call for a cleanup.****

** **

Look. It does not do that. Ever. In any universe. I don't know what you think you are doing,****

but it is definitely not what you wrote. Look at the code. It can't do that.****

** **

matt****

****

To be sure, I tried the whole thing in a new ARCH with a fresh****

configure step, and this time builder.py was not used at all (by me)****

but it led to the same issues with builder2.py.****

****

I understand that this is not the most used platform, so let me****

know if I can be of help. And I forgot to mention that the****

python build was 4-5 times faster than “ar” based build and****

that’s great...****

****

Chetan****

****

****

****

****

****

*From:* [email protected] [mailto: [email protected]] *On Behalf Of *Matthew Knepley *Sent:* Thursday, June 28, 2012 3:21 PM *To:* For users of the development version of PETSc *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin****

****

On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <[email protected]> wrote:****

Matt,****

****

I was surprised too on both counts. Couldn’t find any libpetsc.lib after the python build****

and I did not run “clean” in between that could have deleted the .o files. ****

****

Here’s one more try below to show it.****

****

*$ ls test_8/lib/*****

libpetsc-obj****

****

*$ ls test_8/lib/libpetsc-obj/ | wc -l*****

*560*****

****

*$ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py*****

*(The same happens (existing .o are deleted) if ./config/builder2.py build is used)*****

****

This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options,****

so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it.****

****

I said to use builder2.py, which does proper dependency tracking.****

****

Matt****

****

* *****

Rebuilding Dependencies****

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys****

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag****

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm****

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll****

****

*I stopped the build using control-c. It would have deleted all files first.*****

****

Traceback (most recent call last):****

File "./config/builder.py", line 1534, in <module>****

PETScMaker().run()****

File "./config/builder.py", line 1526, in run****

if self.buildLibraries('libpetsc', self.rootDir):****

File "./config/builder.py", line 1331, in buildLibraries****

objects += self.buildDir(root, files, objDir)****

File "./config/builder.py", line 1251, in buildDir****

objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir))****

File "./config/builder.py", line 1025, in compileC****

return self.compile(self.configInfo.languages.clanguage, source, objDir)****

File "./config/builder.py", line 1012, in compile****

(output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log)****

File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand****

(output, error, status) = runInShell(command, log, cwd)****

File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell****

thread.join(timeout)****

File "/usr/lib/python2.6/threading.py", line 655, in join****

self.__block.wait(delay)****

File "/usr/lib/python2.6/threading.py", line 258, in wait****

_sleep(delay)****

KeyboardInterrupt****

****

*$ ls test_8/lib/libpetsc-obj/*****

*bag.o*****

****

All the previous .o were deleted. Perhaps I’m not doing something right?* ***

****

Chetan****

****

****

*From:* [email protected] [mailto: [email protected]] *On Behalf Of *Matthew Knepley *Sent:* Thursday, June 28, 2012 2:46 PM *To:* For users of the development version of PETSc *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin****

****

On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <[email protected]> wrote:****

Thanks Matt, Satish. A few issues though.****

****

Like you said, the latest cygwin has python 2.6.8 which****

does not have argparse. When I place the following****

in path http://argparse.googlecode.com/svn/trunk/argparse.py****

I can run the python build with 2.6.8.****

****

However, the “ar” based build had worked fine and the python****

based build fails for these two files.****

****

src/snes/impls/MULTIB~1/multiblock.c****

src/snes/impls/vi/rsaug/virsaug.c****

****

Yep, it fails, but the build succeeds, so no problem.****

****

It looks like these two files were NOT compiled in the ar based****

build. The two make.log files are attached.****

****

One more question. I tried to run the python based build****

the second time (after it was unsuccessful the first time)****

and then it deleted all the previous .o files it had created in ****

ARCH/lib/libpetsc-obj. How can that be avoided?****

****

It does not do that unless you run 'clean'.****

****

Matt****

****

Chetan****

****

****

*From:* [email protected] [mailto: [email protected]] *On Behalf Of *Matthew Knepley *Sent:* Thursday, June 28, 2012 1:35 PM *To:* For users of the development version of PETSc *Subject:* Re: [petsc-dev] slow "ar Scq" on cygwin****

****

On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <[email protected]> wrote:****

Hi devs,

Is there a way to avoid running the archiver (ar) every time a petsc directory is compiled?

The reason I'm asking this is that when libpetsc.lib becomes large enough (say 100MB+, esp. for debug build), "ar Sqc" on cygwin takes very long in each directory. Depending on OS caching, very long can be 10 seconds per PETSc directory just to run ar.

It creates a new temp file from scratch, puts stuff from libpetsc.lib and new stuff in it and then renames the temp file. As you can imagine, the compilation process is fast in the beginning but crawls by the end. I've seen the same IO sequence on Linux (with Sqc flags to ar), but since cygwin IO is slower the effect is more clearly visible. Building within a ramdisk leads to 25% faster compilation but it was more of a curiosity.****

****

Build with Python. It never uses antique things like ar :)****

****

python2.7 ./config/builder2.py build****

****

Matt****

****

Chetan****

****

****

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener****

****

****

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener****

****

****

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener****

****

** **

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener****

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener