February 2010 - Nek5000-users - mailman3.cels.anl.gov

LES and heat transfer in channel
by nek5000-users＠lists.mcs.anl.gov 21 Feb '10

21 Feb '10

Hi, I am running a channel LES simulation (periodic in streamwise and spanwise, length=width=9*channel height) in nek with the following parameters: -LES model: Filter last 3 modes, 5% filter weight; lx1=14; ld1=20 -Piece of the rea file " 1.00000 DENSITY 0.31250E-04 VISCOS 1.00000 RHOCP 1.00000 CONDUCT 4.00000 p99=3 ==> dealiasing turned on T IFFLOW T IFHEAT T IFTRAN T T F F F F F F F F F IFNAV & IFADVC (convection in P.S. fields) F F T T T T T T T T T T IFTMSH (IF mesh for this field is T mesh) F IFAXIS F IFSTRS F IFSPLIT F IFMGRID F IFMODEL F IFKEPS F IFMVBD T IFCHAR " -Reynolds number based on 2*channel height: 12800 -Constant heat flux on walls -The streamwise boundary is a recycling one, I set it up similar to the turbJet example, but I added temperature recycling. To avoid "over-heating", I subtract the added thermal energy from the temperature when recycling -Some material parameters are set in the .usr file: " param(8) = param(2)/0.71 ! Prandt=0.71 cpfld(2,1) = param(8) ! conductivity " When I compare the spanwise averaged and time averaged Nusselt number on a wall, it is fairly constant with streamwise direction (which is good), but about 50% higher then the experimental one from the correlation Nu=0.022*Re^0.8*Pr^0.5 This might be because I am only 2.5 flow throughs away from the initial condition, but I wanted to make sure that everything is right before putting in more computing hours. Does the above look OK? Is there another explanation for the heat transfer coefficient being too high? Thanks, Markus

1 3

Segmentation fault occuerd
by nek5000-users＠lists.mcs.anl.gov 15 Feb '10

15 Feb '10

Hello, I have some trouble to run my simulation... When I start running my simulation, I receive the following error-message (I compiled the run in debugging mode -g and with traceback): forrtl: severe (174): SIGSEGV, segmentation fault occurred Image PC Routine Line Source libc.so.6 00002B76C2128FA0 Unknown Unknown Unknown nek5000 0000000000500D48 chcopy_ 434 math.f nek5000 000000000071BA91 exitti_ 399 comm_mpi.f nek5000 0000000000565F6A set_conv_char_ 603 convect.f nek5000 0000000000561C80 setup_convect_ 44 convect.f nek5000 00000000004E805C setics_ 393 ic.f nek5000 0000000000417B2A nek_init_ 164 drive1.f nek5000 0000000000415FCD MAIN__ 26 drive.f nek5000 0000000000415F8C Unknown Unknown Unknown libc.so.6 00002B76C20D14CA Unknown Unknown Unknown nek5000 0000000000415EBA Unknown Unknown Unknown -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 10816 on node ifh-taifun exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- By debbuging I've seen, that follwoing values in the subroutine set_conv_char (in convect.f) seem to be a little strange: nxd= 0 nyd= 7457184 nzd= 0 In my DEALIAS-file the follwing line is commented out: C common /dedim/ nxd,nyd,nzd I tried to run the simulation with the declaration in the DEALIAS-file and a include 'DEALIAS' in the subroutine set_conv_char (in convect.f) but the the run also is dying: ZWGJD: Minimum number of Gauss points is 1 -1633391248 call exitt: dying ... Do you maybe know where there might be my buck?? Thx a lot. Fred Here my SIZEu-file: C Dimension file to be included C C HCUBE array dimensions C PARAMETER (LDIM=3) PARAMETER (LX1=6,LY1=6,LZ1=6,LELT=4420,LELV=4420) parameter (lxd=9,lyd=lxd,lzd=lxd) parameter (lelx=6,lely=lelx,lelz=lelx) PARAMETER (LZL=3) PARAMETER (LX2=LX1-2) PARAMETER (LY2=LY1-2) PARAMETER (LZ2=LZ1-2) PARAMETER (LX3=LX1) PARAMETER (LY3=LY1) PARAMETER (LZ3=LZ1) parameter (lpelv=1,lpelt=1,lpert=1) ! perturbation parameter (lpx1=1,lpy1=1,lpz1=1) ! array sizes parameter (lpx2=1,lpy2=1,lpz2=1) parameter (lbelv=1,lbelt=1) ! MHD parameter (lbx1=1,lby1=1,lbz1=1) ! array sizes parameter (lbx2=1,lby2=1,lbz2=1) C LX1M=LX1 when there are moving meshes; =1 otherwise PARAMETER (LX1M=1,LY1M=1,LZ1M=1) PARAMETER (LDIMT= 1) c PARAMETER (LDIMT= 3) PARAMETER (LDIMT1=LDIMT+1) PARAMETER (LDIMT3=LDIMT+3) PARAMETER (LP = 8) PARAMETER (LELG = LP*LELT) c c c Note: In the new code, LELGEC should be about sqrt(LELG) c PARAMETER (LELGEC = 1) PARAMETER (LXYZ2 = 1) PARAMETER (LXZ21 = 1) c PARAMETER (LMAXV=LX1*LY1*LZ1*LELV) PARAMETER (LMAXT=LX1*LY1*LZ1*LELT) PARAMETER (LMAXP=LX2*LY2*LZ2*LELV) PARAMETER (LXZ=LX1*LZ1) PARAMETER (LORDER=3) C FF PARAMETER (MAXOBJ=2,MAXMBR=LELT*6) PARAMETER (MAXOBJ=4,MAXMBR=LELT*6,lhis=100) C C Common Block Dimensions C PARAMETER (LCTMP0 =2*LX1*LY1*LZ1*LELT) PARAMETER (LCTMP1 =4*LX1*LY1*LZ1*LELT) C C The parameter LVEC controls whether an additional 42 field arrays C are required for Steady State Solutions. If you are not using C Steady State, it is recommended that LVEC=1. C PARAMETER (LVEC=1) C C Uzawa projection array dimensions C C FF PARAMETER (MXPREV = 10) parameter (mxprev = 20) parameter (lgmres = 40) C C Split projection array dimensions C parameter(lmvec = 1) parameter(lsvec = 1) parameter(lstore=lmvec*lsvec) c c NONCONFORMING STUFF c parameter (maxmor = lelt) C C Array dimensions C COMMON/DIMN/NELV,NELT,NX1,NY1,NZ1,NX2,NY2,NZ2 $,NX3,NY3,NZ3,NDIM,NFIELD,NID c automatically added by makenek parameter(lxo = lx1) ! max output grid size (lxo>=lx1) c automatically added by makenek parameter(lpart = 1 ) ! max number of particles c automatically added by makenek integer ax1,ay1,az1,ax2,ay2,az2 parameter (ax1=lx1,ay1=ly1,az1=lz1,ax2=lx2,ay2=ly2,az2=lz2) ! running averages

1 1

Domain boundary visualization in VisIt
by nek5000-users＠lists.mcs.anl.gov 04 Feb '10

04 Feb '10

Hi all, hoping not to have found a solution to a non-existing problem, I'd like to post my idea about how to visualize domain boundaries based on their type in VisIt. That'll come in handy when I want to generate a picture to describe with colors/gray levels what boundary conditions were used in the simulation. First, I add the following lines to the .usr file: " real bcscal(lx1,ly1,lz1,lelt) !Scalar for boundary condition visualization integer e, f ntot = nx1*ny1*nz1*nelv call rzero(t(1,1,1,1,2),ntot) call rzero(bcscal,ntot) c-----Set scalar values for different boundaries nface = 2*ndim do e=1,nelv do f=1,nface if (cbc(f,e,1).eq.'P ') then call facev(bcscal,e,f,1.0,lx1,ly1,lz1) endif if (cbc(f,e,1).eq.'O ') then call facev(bcscal,e,f,2.0,lx1,ly1,lz1) endif if (cbc(f,e,1).eq.'W ') then call facev(bcscal,e,f,3.0,lx1,ly1,lz1) endif if (cbc(f,e,1).eq.'v ') then call facev(bcscal,e,f,4.0,lx1,ly1,lz1) endif enddo enddo call copy(t(1,1,1,1,2),bcscal,ntot) " This will assign different values (1 for periodic, 2 for outflow, ...) to faces on boundaries of certain types. " ifpsco(1) = .true. ifreguo = .false. call prepost(.true.,'bcs') " This will turn on output of scalar 1 (t(1,1,1,1,2)) and write it out on a GLL mesh with filename prefix 'bcs'. In VisIt, I then define equations (Controls->Expressions...) that create new scalar variables (wall, periodic, ...), for example for walls: Variable name: walls " if(eq(s1,3), walls=1, walls=0) " I then use a "Contour" plot for each variable. I'd be glad to hear some feedback, Markus

1 1

Conjugate Heat Transfer
by nek5000-users＠lists.mcs.anl.gov 04 Feb '10

04 Feb '10

Hi, I have experienced some difficulties with setting up a 2D conjugate heat transfer problem. When generating the .rea file with prenek, I create a mesh of, say 60 elements and define, say, 20 of those as solids (material group 1 instead of 0). Trying to run this .rea file in genmap (even when it is converted to re2) does not work since the fluid boundary condition section of the .rea file contains entries for elements (the solids) without connectivity/boundary information. Deleting those entries (4*20 lines for 2D) lets genmap accept the rea file and a map is created with one section for the fluids elements and one for the solids. Running these files with nek5000, however, resulted in element mismatch errors, no matter what combination of solid elements vs. number of processors I tried. The newest nek5000 svn was used. The solution I found is the following: when one takes the .rea file for conjugate heat transfer (as it came out of prenek) and changes all solid elements back to fluids, a map can be generated from this only-fluids-.rea file with genmap that is essentially left in ignorance of the conjugate heat transfer problem/solid elements. Running this "fluids-only" map with the original "conjugate heat"-.rea file (having deleted the solid elements boundary/connectivity information in its fluids section) will work and produces physically correct looking results. My questions now are the following: -Is this an appropriate way to go or am I missing some parameters that need to be set? The rea files come directly out of prenek, so I didn't manually change any parameters. -By generating a map file that does not resemble what's truly going on, do we loose parallel scalability and if so, is that a considerable penalty? -I had some trouble defining solid elements in 3D with prenek. Whenever I click on an element to change its material group, it does not do it. Is there a fix? -Expanding a 2D conjugate heat transfer problem with n2to3 does not work. All upper level elements become solids. Is there a fix for this? I apologize for my lengthy postings, Markus

1 2

Re: [Nek5000-users] Nek5000-users Digest, Vol 12, Issue 1
by nek5000-users＠lists.mcs.anl.gov 03 Feb '10

03 Feb '10

Hi Paul, It certainly helps me in setting the parameters lelt and lelg. Thanks. Just curious, is there a minimum number of processors that I need to have for running a simulation with 120k elements ? For example, I was not able to go more than lx1 of 7 on cluster with 128 processors while on other machine I could go upto lx1 = 12 ( With 128 processors) . Thanks Shriram

1 2

Re: [Nek5000-users] Nek5000-users Digest, Vol 12, Issue 1
by nek5000-users＠lists.mcs.anl.gov 03 Feb '10

03 Feb '10

Hi Stefan, I don't have the log file with me as we later submitted the job to a supercomputer. I shall post it if I get to notice the same problem again. Thanks Shriram

1 1

Number of processors
by nek5000-users＠lists.mcs.anl.gov 03 Feb '10

03 Feb '10

Hallo, just a short quastion: How can I define the number of processors?? I only found in the SIZEu-file the parameter LP, for the maximum number of processors.. Thx a lot. Fred

1 1

Scalability
by nek5000-users＠lists.mcs.anl.gov 02 Feb '10

02 Feb '10

Hi, When I am trying to run a big simulation (Nele > 120,000) , on certain clusters / supercomputers, Nek5000 crashes even though the physical memory used is very less. For example, on certain machines, with Nele = 120,000, we couldn't go more than lx1 of 7 while on different architecture we could go up to lx1 = 12. Is this issue concerned with compilers or the architecture of the machine ? Thanks Shriram Jagannathan Texas A&M University

1 2