December 2016 - Nek5000-users - mailman3.cels.anl.gov

double mesh ?
by nek5000-users＠lists.mcs.anl.gov 16 Jan '17

16 Jan '17

Hi i am quite new, so i’m sorry if i’m getting things wrong. I am trying to simulate a turbulent channel flow, with a passive scalar. in the streamwise direction, i would like to impose periodic boundary conditions for the flow, and non-periodic for the scalar. in fact the scalar has a point source near the inlet, and i want to see how that develops downstream. as far as i understand this is not supported by nek5000, because the last mesh point is identified with the first mesh point when periodic BC are imposed. correct ? my question is : could i define two meshes, one for the scalar and one for the velocity ? thanks a lot for your help !!! agnese Agnese Seminara -------------------------------- CNRS Laboratoire de physique de la matière condensée Parc Valrose avenue J Vallot 06108 Nice, France +33 (0) 492 076 775 http://sites.unice.fr/site/aseminara/ <http://sites.unice.fr/site/aseminara/>

1 2

Compiling problem in Nek5000 tools
by nek5000-users＠lists.mcs.anl.gov 27 Dec '16

27 Dec '16

Dear Nek5000 users, I'm a new user. When I was compiling Nek5000 tools as Nek5000 Manual, I met a problem as below. And I surf the Internet for help and found that I should add "-mcmodel=medium" in makefile. But I don't know how to deal with this. Can anyone help me to solve this problem? Thanks a lot! after ./maketools all I got this Best wishes! Weicheng Hu December 24th, 2016

1 4

Problem using genmap with large numbers of elements
by nek5000-users＠lists.mcs.anl.gov 22 Dec '16

22 Dec '16

Dear Nek Users, We are about to set up a new case with about 29692 in 2D, which we would like to extrude to about 150 elements in 3D, giving a total of about 30k times 150 = 4.5M elements. n2to3 works fine, but genmap gives a "periodic mismatch error". Having a lower number of extrusion elements, say 30k times 70 = 2.1M elements, works fine. Changing the tolerance in genmap does not fix it. Before diving into the code of n2to3 and genmap, we wanted to ask whether somebody has experienced that before, and whether you'd know a potential fix for the problem. We used both an older version of genmap and the latest github version. Thanks a lot! Ricardo

1 5

Mixed boundary condition for fluid velocity
by nek5000-users＠lists.mcs.anl.gov 19 Dec '16

19 Dec '16

Dear Nek users and developers, I am working on turbulent flow over porous walls. Instead of no slip condition for rigid walls, I need to implement the following boundary condition for the fluid velocity on the porous walls u = c1 Sigma . N + F(Sigma) where, c1 — a scalar constant u — fluid velocity Sigma — fluid stresse tensor N — Normal vector F(Sigma) — a simple linear function of fluid stresses. If F(Sigma)=0, then the implementation is straightforward (similar the convection boundary condition implementation for heat transfer problem). It is not clear to me how to incorporate F(Sigma). Could someone explain me how can I add an unknown term (involving surface integration) into the boundary conditions? I have already looked into the code but could not understand where the surface integral coming out of the integration by parts (in the stress formulation) is added to the system matrix in the code. Could someone also point me out where this is done? Thank you very much, Best regards, Sudhakar KTH-Mechanics.

1 0

Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2
by nek5000-users＠lists.mcs.anl.gov 15 Dec '16

15 Dec '16

Dear Stefan, Thank you so much for your detailed explanation! Now I understand the scaling. Best regards, Wei XU

1 0

Project from the velocity grid to the pressure grid.
by nek5000-users＠lists.mcs.anl.gov 14 Dec '16

14 Dec '16

Hi Neks, I am trying to extrude 2D fields .fld to 3D fields. To do so I followed what was said there http://lists.mcs.anl.gov/pipermail/nek5000-users/2013-October/002333.html . It worked fine for the velocities and the temperature fields, but for the pressure, I am having some troubles. I am using the PN/PN-2 approach, and so my velocities and pressure are not defined on the same grid, but the pressure values that I can recover from the .fld are mapped on the velocity grid. My question then would be: is there a (quick) way to have the pressure "projected back" on its grid ? Did anyone has this problem ? Thank you for your help, Pierre-Emmanuel des Boscs.

1 0

Re: [Nek5000-users] maketools all fails at postnek on RHEL6.4
by nek5000-users＠lists.mcs.anl.gov 14 Dec '16

14 Dec '16

Using the tarball at https://github.com/Nek5000/nek5000/archive/master.tar.gz to compile locally on my RHEL6.4 server. During maketools event, only the genmap and n2to3 tools are compiled and the make event fails out as shown below. Is this attributed to BIGMEM="true" in maketools, or something else? Can anyone help? Error and gcc/gfortran information below... Thanks Richard P [user@rhel64 tools]$ gedit maketools [user@rhel64 tools]$ ./maketools all /nfs/home/user/Nek5000 ---------------------- Make genmap... ---------------------- make[1]: Entering directory `/home/user/Nek5000-master/tools/genmap' gfortran -mcmodel=medium -fdefault-real-8 -o /nfs/home/user/Nek5000/genmap genmap.o byte.o make[1]: Leaving directory `/home/user/Nek5000-master/tools/genmap' ---------------------- Make n2to3... ---------------------- make[1]: Entering directory `/home/user/Nek5000-master/tools/n2to3' gfortran -mcmodel=medium -o /nfs/home/user/Nek5000/n2to3 n2to3.o byte.o make[1]: Leaving directory `/home/user/Nek5000-master/tools/n2to3' ---------------------- Make postnek... ---------------------- make[1]: Entering directory `/home/user/Nek5000-master/tools/postnek' gfortran -mcmodel=medium -c postnek.f postnek.f:1977.19: ifbswap = if_byte_swap_test(test) 1 Warning: Extension: Conversion from INTEGER(4) to LOGICAL(4) at (1) gfortran -mcmodel=medium -c postnek2.f gfortran -mcmodel=medium -c postnek3.f postnek3.f:1552.20: COMMON /CTMP4/ wk(LXYZ,3,3) 1 Warning: Named COMMON block 'ctmp4' at (1) shall be of the same size gfortran -mcmodel=medium -c postnek5.f postnek5.f:711.21: COMMON /ccdiag/ iee,iii 1 Warning: Named COMMON block 'ccdiag' at (1) shall be of the same size gfortran -mcmodel=medium -c postnek6.f gfortran -mcmodel=medium -c tsort.f gfortran -mcmodel=medium -c postnek8.f devices.inc:22.19: Included at basics.inc:168: Included at postnek8.f:96: COMMON/SCALE/XFAC ,YFAC ,XZERO ,YZERO 1 Warning: Named COMMON block 'scale' at (1) shall be of the same size gfortran -mcmodel=medium -c postnek9.f gfortran -mcmodel=medium -c plot.f gfortran -mcmodel=medium -c getfld.f gfortran -mcmodel=medium -c legend.f gfortran -mcmodel=medium -c userf.f userf.f:1282.21: common /rlobjs/ x_l_objs(mopts),y_l_objs(mopts),z_l_objs(mopts) 1 Warning: Named COMMON block 'rlobjs' at (1) shall be of the same size userf.f:1816.21: common /rlobjs/ avg(6,0:mopts-1,3) 1 Warning: Named COMMON block 'rlobjs' at (1) shall be of the same size userf.f:5261.20: common /ctmp1/ uavg(nxm*nelm),vavg(nxm*nelm),wavg(nxm*nelm) 1 Warning: Named COMMON block 'ctmp1' at (1) shall be of the same size gfortran -mcmodel=medium -c trap.f gfortran -mcmodel=medium -c animate.f gfortran -mcmodel=medium -c genxyz.f gfortran -mcmodel=medium -c screen.f gfortran -mcmodel=medium -c g3d.f gfortran -mcmodel=medium -c subs.f gfortran -mcmodel=medium -c xinterface.f gfortran -mcmodel=medium -c locglob.f gfortran -mcmodel=medium -c postnek5a.f gfortran -mcmodel=medium -c ../../nek/3rd_party/blas.f gfortran: ../../nek/3rd_party/blas.f: No such file or directory gfortran: no input files make[1]: *** [blas.o] Error 1 make[1]: Leaving directory `/home/user/Nek5000-master/tools/postnek' make: *** [all] Error 1 ###########gcc and gfortran details for server########## [user@rhel64 tools]$ rpm -qa | grep gcc gcc-4.4.7-3.el6.x86_64 compat-gcc-34-3.4.6-19.el6.x86_64 gcc-gfortran-4.4.7-3.el6.x86_64 libgcc-4.4.7-3.el6.i686 gcc-c++-4.4.7-3.el6.x86_64 libgcc-4.4.7-3.el6.x86_64 [user@rhel64 tools]$ rpm -qa | grep gfortran gcc-gfortran-4.4.7-3.el6.x86_64 libgfortran-4.4.7-3.el6.x86_64 compat-libgfortran-41-4.1.2-39.el6.x86_64 Richard B Powell Technical Architect II Operations/Server Engineering AREVA, Inc. 3315 Old Forest Road OF-60 Lynchburg, VA 24501 Phone: +1 434-832-3894 Fax: +1 434-382-3894

1 5

(no subject)
by nek5000-users＠lists.mcs.anl.gov 14 Dec '16

14 Dec '16

This issue seems to encompass the problem you're having. It also addresses how to fix it. https://github.com/Nek5000/Nek5000/issues/84 Hope this helps, Lane

1 0

How to call mpi_barrier in usrcheck
by nek5000-users＠lists.mcs.anl.gov 14 Dec '16

14 Dec '16

Hi Neks, I am currently facing a problem where I need to synchronize all my processors when I'm reading and writing files. Otherwise I would get an error of type 'end of file'. Can someone explain to me how to use the command call MPI_BARRIER with an example of syntax that I would integrate to the usr check. Thank you in advance, Best regards, Arnold *Arnold Wakim* Doctorant, Département d´Aérodynamique Fondamentale et Expérimentale Tél: +33 1 46 23 51 83 ONERA - The French Aerospace Lab -Centre de Meudon 8, rue des Vertugadins - 92190 MEUDON Nous suivre sur : www.onera.fr <http://www.onera.fr/> | Twitter <http://www.twitter.com/@onera_fr> | LinkedIn <http://www.linkedin.com/company/onera> | Facebook <http://www.facebook.fr/thefrenchaerospacelab>

1 2

Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2
by nek5000-users＠lists.mcs.anl.gov 13 Dec '16

13 Dec '16

Typically people look at inter-nocde scaling when they talk about parallel performance. Intra-node scaling (the case you look at) is a different story. Let’s consider two extremes: 1. STREAM triad benchmark is purely memory bound. On E5-2570v3 (Intel Haswell) you’ll get ~40 GB/sec using 2 cores (1 out of 18 cores per socket). If this would scale in a linear way we should see ~720GB/s using all 36 cores. However, in reality you'll get an aggregated peak bandwith ~90/sec. So it does _not_ scale linearly. In fact it levels off after 12 cores (6 cores per socket). 2. DGEMM benchmark is purely memory bound. On E5-2570v3 (Intel Haswell) you’ll get ~80% total peak floating points performance using all 36 cores. It scales more or less in a linear way with the number of cores. Nek5000 (like _all_ other PDE solvers) is somewhere in between. On most systems we'll get 5-20% of peak depending on the hardware architecture and polynomial order. Compare this with other PDE based solvers. Cheers, Stefan -----Original message----- > From:[email protected] <nek5000-users(a)lists.mcs.anl.gov> > Sent: Tuesday 13th December 2016 11:48 > To: nek5000-users(a)lists.mcs.anl.gov > Subject: Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2 > > Dear Stefan, > > Sorry, I didnt make myself clear. What I meant is whether we can further improve the parallel computing performance, as I am only using about 1/3 of the computing power of each node. Thank you very much! > > Best regards, > Wei XU > > Date: Wed, 07 Dec 2016 12:01:42 +0100 From: nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> To: <nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Subject: Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2 Message-ID: <mailman.7291.1481108507.3602.nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Content-Type: text/plain; charset="utf-8" Why do think something is wrong i.e. you have a problem? From: <nek5000-users-bounces(a)lists.mcs.anl.gov <mailto:ne k5000-us ers-bounces(a)lists.mcs.anl.gov>> on behalf of <nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Reply-To: <nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Date: Wednesday, December 7, 2016 at 10:39 AM To: <nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Subject: Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2 Dear Stefan, Thank you very much! Can I do anything on the Nek5000 side to reduce the problem? Best regards Wei XU Date: Tue, 6 Dec 2016 16:34:04 +0100 From: nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> To: nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> Subject: Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2 Message-ID: <mailman.7242.1481038500.3602.nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Content-Type: text/plain; charset="us-ascii" Thata reasonable. The core compete for shared resources (L3 and DDR). _______________________________________________ Nek5000-users mailing list Nek5000-users@lists. mcs.anl. gov <mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users <https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.mcs.anl.gov/pipermail/nek5000-users/attachments/20161207/c5b7a… <http://lists.mcs.anl.gov/pipermail/nek5000-users/attachments/20161207/c5b7a…>> ------------------------------ > > Date: Tue, 6 Dec 2016 16:25:22 +0800 From: nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> To: nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> Subject: Re: [Nek5000-users] Parallel speedup on supercomputer Tianhe-2 Message-ID: <mailman.7221.1481012750.3602.nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> Content-Type: text/plain; charset="utf-8" Dear Stefan, Thank you for your reply. There are 139056 elements and the polynomial order is 7 (lx1=8). I measure the solver time to compute the speedup. For example, the serial job takes 2564.59s and the same job with 24 tasks takes 302.18s. The speedup is about 8.5. This is on a single Tianhe-2 node. The speedup between nodes is quite good. I also test the code on my 36-core computer (Dual Xeon E5 18-Core). I can only get about 12 times speedup when I use 36 tasks. It is also about 1/3. Best regards, Wei XU > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

1 0