October 2010 - petsc-users - mailman3.cels.anl.gov

Re: Multiple communicators
by irfan.khan＠gatech.edu 02 Apr '12

02 Apr '12

I use seperete communicators for fluid and solid ranks in a fluid-structure interaction code. I use PETSc tools (KSP) to solve for the solid phase (FEA) which is carried out under a different communicator (FEA_Comm). Unless I am doing something wrong, I have found that the command line options: -ksp_type; -mat_partitioning_type, don't work if there are 2 communicators. But for single communicator they work. Please see the attached files containing output of -log_summary for 2 different codes (scroll down to the end). One with single communicator and the other with two communicators. Both the output have been generated with the same PETSc compilation. Futher -log_summary results are not able to detect all the events that have been registered if they are not in the WORLD communicator. Thank you Irfan ----- Original Message ----- From: "Matthew Knepley" <knepley(a)gmail.com> To: "PETSc users list" <petsc-users(a)mcs.anl.gov> Sent: Tuesday, November 24, 2009 8:03:00 PM GMT -05:00 US/Canada Eastern Subject: Re: Multiple communicators On Tue, Nov 24, 2009 at 6:59 PM, < irfan.khan(a)gatech.edu > wrote: Hello Does the procedure for providing command line options and to obtain profiling data through -log_summary change if there are multiple MPI communicators in a petsc code? There is always WORLD, so it should be fine unless I misunderstand your question. Matt If so, can somebody point me to information, references on how to provide command line options if there are multiple communicators? Thank you Irfan -- PhD Candidate G.W. Woodruff School of Mechanical Engineering Georgia Institute of Technology Atlanta, GA (30332) -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- PhD Candidate G.W. Woodruff School of Mechanical Engineering Georgia Institute of Technology Atlanta, GA (30332)

5 14

parmetis weights
by francois pacull 09 Aug '11

09 Aug '11

Dear PETSc team, I just have a small and naive question about graph weights in Parmetis. When looking at mat/partition/impls/pmetis/pmetis.c, it seems that weight arrays (for vertices or edges) can be given to parmetis, however the wgtflag variable is always set to 0 from what I understand, which means that no weights are used in ParMETIS_V3_PartKway. So does this mean that the parmetis routine uses the weight arrays whenever they are not NULL, even if wgtflag is 0? I didn't try to use any weight yet, but will probably try it soon... Thank you for any help you can provide, francois pacull.

2 2

V-cycle multigrid with matrix shells
by Sylvain Barbot 19 May '11

19 May '11

Hi, I am trying to implement the multigrid solver to solve a linear inhomogeneous elliptic equation with non-constant coefficients in 3-D with 3 degrees of freedom with matrix shells. The grid is structured with non-uniform sampling. I insist on using matrix shells because of memory limitations. The direct solver gmres, or alike, gives very accurate solutions even for large problems (say, with a grid of 256x256x256), but computation is lengthy. I am hoping to get significant speedup using a multigrid solver. It looks like the DMMG objects may provide all the tools necessary to do so: smoother, restriction and interpolation. But typically in my problems, I do not have an analytical expression for the right-hand side, just a given forcing term in finest resolution with high-frequency content (not easily subsampled). Also I do not typically have any non-trivial first guess. It looks the the default Petsc multi-grid solver starts from the coarser grid and builds a solution with a finer and finer resolution as levels increase. I would like to do the opposite, with the typical smoothing of the residuals, then restriction, smoothing, restriction, then direct solving at the coarsest level, then as many interpolation-correction-smoothing as necessary to go back to the finest level, the typical V cycle. questions: 1) is it at all possible to specify this mode of operation, from the finest to the coarser level, and back? Any examples out there? 2) is it readily possible to use matrix shells with DMMG? I imagine the Jacobian matrix may simply be provided as a matrix shell. Is there any examples of multi-grid methods with shell matrices online? 3) to deal with non-uniform sampling: can I provide the coordinates of the finest grid with DASetCoordinates, then expect DMMG to provide the subsampled coordinates at the coarser levels? Thanks a lot in advance for any advice. Best, Sylvain Barbot

6 28

Some basic questions about PETSC
by Ben Tay 19 Jan '11

19 Jan '11

Hi, I'm still relatively new to PETSC but I've finally managed to solve my own poisson eqn using it in about a week's time.Thank you for writing such a good solver. However, I have some questions still about PETSC: 1. Are petscscaler the same as real nos.? I am writing in double precision or real(8). So are these 2 interchangeable? (same as petscint and integer). 2. while linking in compaq visual fortran, I get the warning message about conflict library with LIBCMT. Is it important or can I just ignore it? 3. I am writing in f90 free form format. When I modify from fixed f77 format to f90, it gives me a lot of error, which seems to be the include file error. So, is it possible to write in f90 format? 4. I am writing a Navier stokes solver and I'm using PETSC to solve the poisson eqn. If only the RHS changes with each time step, do I need to call VecCreateSeq <insert values in RHS b> VecAssemblyBegin VecAssemblyEnd VecDestroy at each time step? I don't think I can (or I should) change values of b after VecAssemblyBegin,VecAssemblyEnd, or can I? Thank you very much and have a nice day Regards, Ben

4 8

Writing PETSc matrices
by Nunion 13 Dec '10

13 Dec '10

Hello, I am new to PETSc and programming. I have a question concerning writing PETSc matrices in binary from binary matrices [compressessed/uncompressed] generated in Matlab. I am attempting to use the files in the /bin/matlab directory, in particular the PetscBinaryWrite.m file. However, the usage; PetscBinaryWrite('matrix.mat','output.ex') does not seem to work. I also tried using the examples in the /mat directory however, matlab does not support the writing of complex matrices in ASCII. Thanks in advance, Tom

5 6

Packages Similar to PETSc in C++?
by Mohammad Mirzadeh 31 Oct '10

31 Oct '10

Dear all, I was wondering if anyone can refer me to a package similar to PETSc but that is written in c++? Right now I have a large code written in c++ for doing CFD simulations that I need to transform from serial to parallel. Initially I was thinking of PETSc and tried using it but found that PETSc is written in C and thus does not allow to have arrays (in parallel) of arbitrary type. I have a big data structure and it is much easier for me to retain the current structure and form of the code. As a result I was wondering if you guys know of any similar package in C++ ?(in the sense that it can provide with efficient linear solvers in parallel while hiding most of MPI from the user) I could think of HYPRE but then again I am not sure it is written in C++. Thanks, Mohammad

7 8

Problem with mpi
by TAY Wee Beng 29 Oct '10

29 Oct '10

Hi, I have a mpi code which works fine in my previous clusters. However, there's mpi problem when I use it in the new clusters, which use openmpi. I wonder if there's a relation. The error is: [n12-70:14429] *** An error occurred in MPI_comm_size [n12-70:14429] *** on communicator MPI_COMM_WORLD [n12-70:14429] *** MPI_ERR_COMM: invalid communicator [n12-70:14429] *** MPI_ERRORS_ARE_FATAL (your MPI job will now abort) 2.48user 0.45system 0:03.99elapsed 73%CPU (0avgtext+0avgdata 2871936maxresident)k 0inputs+0outputs (0major+185587minor)pagefaults 0swaps -------------------------------------------------------------------------- mpiexec has exited due to process rank 3 with PID 14425 on node n12-70 exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpiexec (as reported here). -------------------------------------------------------------------------- [n12-70:14421] 3 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal [n12-70:14421] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages I am not sure what other information you will need. I will provide more information if required. I tried running on 1, 4 and 8 processors but all can't work. -- Yours sincerely, TAY Wee Beng

2 1

Problem for PETSc 3.1
by Rongliang Chen 28 Oct '10

28 Oct '10

Hi all, I installed the Petsc 3.1 recently and when I tried to make my program using Petsc 3.1 it said: In file included from /home/rlchen/soft/petsc-3.1-p4/include/petscsys.h:1467, from /home/rlchen/soft/petsc-3.1-p4/include/petscis.h:7, from /home/rlchen/soft/petsc-3.1-p4/include/petscao.h:8, from joab.h:87, from joab.c:54: /home/rlchen/soft/petsc-3.1-p4/include/petscerror.h:81:1: warning: "__SDIR__" redefined <command line>:1:1: warning: this is the location of the previous definition . . . . I never defined the "__SDIR__" and I do not know why it said I redefined "__SDIR__". But I can run my program on Petsc 3.0. Can anyone tell me how to fix this problem? Thank you! Best, Rongliang

2 1

assemble block matrices
by Shao-Ching Huang 27 Oct '10

27 Oct '10

Hi, I need to solve M*x=f, where matrix M consists of 4 blocks: [ A B; C D ] I already have old code to build petsc sparse matrices A, B, C and D and the RHS vectors f1 and f2 (such that f=[f1; f2]), each having global indices starting from 0. My question is: is there already API(s) that I can easily assemble matrices A,B,C and D into M? Or do I have to build M from scratch? Thanks, Shao-Ching

3 3

open source CFD Softwares
by aman singh 24 Oct '10

24 Oct '10

Hello sir, I am Amandeep Singh from IIT Bombay.I am doing a seminar on Open source CFD tools as my M.tech Subject.I want to know about the first release of PETSc and what was its release date?Because i am mentioning this software in one of the CFD tools.It was claimed that OpenFOAM is the first one so i just want to ensure this.I will be thankful to you. -- ---- regards Amandeep Singh Roll No.-10301014 M.tech, Aerospace engg.(Aerodynamics) IIT Powaii,Mumbai

3 3