On Feb 24, 2012, at 2:43 PM, Francis Poulin wrote:
Hello,
I wanted to thank everyone for the help and say that I managed to get it running on the cluster and I have done some efficiency calculations. To run the code I used,
mpirun -np # ./ex45 -da_grid_x 5 -da_grid_y 5 -da_grid_z 5 -da_refine 6 -ksp_monitor -pc_type mg -mg_levels_ksp_type richardson -mg_levels_pc_type sor -log_summary
as suggested and found the following
p(#cpu) Tp (parallel) T1 (serial) Efficiency [ = Tp/(p*T1) ] -------------------------------------------------------------------------------------- 1 904 904 1.00 2 553 904 0.82 4 274 904 0.83 8 138 904 0.82 16 70 904 0.81 32 36 904 0.78
It seems to scale beautifully starting at 2 but there is a big drop from 1 to 2. I suspect there's a very good reason for this, I just don't know what.
You need to understand the issues of memory bandwidth shared between cores and between processors, memory affinity and thread affinity ("binding") see http://www.mcs.anl.gov/petsc/documentation/faq.html#computers Barry
Thanks again for all of your help, Francis
On 2012-02-24, at 12:59 PM, Jed Brown wrote:
On Fri, Feb 24, 2012 at 11:49, Francis Poulin <[email protected]> wrote: I don't seem to have the hydra in my bin folder but I do have petscmpiexec that I've been using.
That's just the name of my mpiexec. Use whichever one is used with your build of PETSc.
Use these, it will run the same method and sizes as the options I gave for ex22 before.
mpiexec.hydra -n 2 ./ex45 -da_grid_x 5 -da_grid_y 5 -da_grid_z 5 -da_refine 5 -ksp_monitor -pc_type mg -mg_levels_ksp_type richardson -mg_levels_pc_type sor -log_summary
Also, I want to install PetSc on an SGI machine that I have access to. I have been told that using MPT would give better performance compared to mpich2. When I configure petsc on this server I don't suppose
-with-mpi-dir=/opt/sgi/mpt
the above would work because of the different name. Do you have a suggestion as to what I could try?
It's just the way you launch parallel jobs.