Ah - yes. I did check the difference in petsc code wrt PETSC_HAVE_CHACO_INT_ASSIGNMENT - but didn't check the details inside trilinos/chaco. So the available standalone chaco tarball is version 2.2 but the version included inside seacas is 3.0.0 [with a switch from short to int] We have these hierarchy of packages - and need to figure out a better [install] interface so the same version is used irrespective of the mode of install? Right now - --download-exodusii gets and installs seacas [with only exodusii enabled] So we have: --download-ml --download-chaco --download-zoltan [separate packages] --download-exodusii [installs seacas - and enables only exodusii] --download-trilinos [installs trilinos which includes seacas(exodusii,chaco), ml - zoltan] since seacas is distributed separately and includes exodusii and chaco - have: --download-seacas=1 [with options exodusii=1,chaco=1 by default [or disabled?] --download-seacas=1 --download-seacas=1 --download-seacas-exodusii=0 [only install chaco] --download-seacas-chaco=0 [only install exodusii] [but zoltan, ml are not distributed separately?] Or should we always use trilinos? - if one wants any of the above packages [a huge download]? and enable or disable selected sub-components as desired? Another similar package is SuiteSparse. At some point we had install for only sub-components(cholmod,umfpack) - and then merged them all into --download-suitesparse. Satish On Mon, 23 Apr 2018, Vaclav Hapla wrote:

I can confirm --download-chaco vs --download-trilinos give different chaco partitioning - I can reproduce on my machine. So I made some investigation.

PETSC_HAVE_CHACO_INT_ASSIGNMENT is set only for --download-trilinos. This macro makes difference only at 4 places:

At src/mat/partition/impls/chaco/chaco.c:8 and src/dm/impls/plex/plexpartition.c:1192: #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT) #include <chaco.h> #else /* Older versions of Chaco do not have an include file */ PETSC_EXTERN int interface(int nvtxs, int *start, int *adjacency, int *vwgts, float *ewgts, float *x, float *y, float *z, char *outassignname, char *outfilename, short *assignment, int architecture, int ndims_tot, int mesh_dims[3], double *goal, int global_method, int local_method, int rqi_flag, int vmax, int ndims, double eigtol, long seed); #endif

At src/mat/partition/impls/chaco/chaco.c:68 and src/dm/impls/plex/plexpartition.c:1227: #if defined(PETSC_HAVE_CHACO_INT_ASSIGNMENT) int *assignment; #else short *assignment; #endif

So it seems PETSc is not responsible for the different outputs.

In Trilinos, Chaco is shipped within the Seacas package. The notice at http://gsjaardema.github.io/seacas/#chaco <http://gsjaardema.github.io/seacas/#chaco> says "The short *assignment argument to the interface function has been changed to int *assignment to permit decompositions with more than 32,768 processors." They also say "There are also now a CHACO_VERSION_MAJOR, CHACO_VERSION_MINOR, CHACO_VERSION_PATCH defines in chaco.h." Looking there, the version is 3.0.0.

PETSc's configure --download-chaco gets this one: http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz <http://ftp.mcs.anl.gov/pub/petsc/externalpackages/Chaco-2.2-p2.tar.gz> It actually includes a git repo. There we can see it's the original author's version + some updates by Barry and Satish, mainly to avoid some name clashes. The original version is probably this one: https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.t... <https://www3.cs.stonybrook.edu/~algorith/implement/chaco/distrib/Chaco-2.2.tar.gz> Looking at the contents of the tarball, these sources data back to 2000. I tried comparing these two version but there are two many changes in formatting to identify crucial changes easily. But definitely these two versions differ and can give different results.

The main intent of my tests in src/dm/impls/plex/examples/tutorials/ex5.c is testing HDF5 I/O with different formats and I just needed to somehow distribute the sequential mesh read from exodus file. So I will replace chaco by simple here. But still it's a question whether we want to do something about the confusing situation with Chaco.

Vaclav

...

22. 4. 2018 v 17:56, Satish Balay <[email protected]>:

On Sun, 22 Apr 2018, Smith, Barry F. wrote:

...

...

It would be good to figure out exactly where the difference comes form [wrt chaco from trilinos vs separate install of chaco - or some other interaction from trilinos]

Different random numbers being generated?

Its trilinos vs non-trilinos build of packages - so I don't think its 'different random number generation' issue.

Trilinos install does:

if self.libraries.check(self.dlib, "interface"): self.addDefine('HAVE_CHACO',1) self.addDefine('HAVE_CHACO_INT_ASSIGNMENT',1) if self.libraries.check(self.dlib, "ML_Set_PrintLevel"): self.addDefine('HAVE_ML',1) if self.libraries.check(self.dlib, "Zoltan_LB_Partition"): self.addDefine('HAVE_ZOLTAN',1) if self.libraries.check(self.dlib, "ex_close"): self.addDefine('HAVE_EXODUSII',1)

For one there is different chaco code in petsc based on this flag HAVE_CHACO_INT_ASSIGNMENT. Its not clear to me if this is causing the difference.

--download-chaco is using 'Chaco-2.2-p2.tar.gz'. This matches the latest chaco tarball from their website [Chaco-2.2.tar.gz]. So I do not know if chaco packaged by trilinos has changes that cause this difference [clearly it added some change that resulted in us using the flag HAVE_CHACO_INT_ASSIGNMENT]

And then there is also exodus.. --download-exodus uses the latest snapshot from https://github.com/gsjaardema/seacas/

Satish