Acme-se

Download

[email protected]

May 2016

2 participants
2 discussions

Transitioning ACME to ESMCI/CIME
by Foucar, James G 25 May '16

25 May '16

Hi Anshu, It looks like your script handles entries being in a different order. That's a good start, but we need much more. I haven't explained to you and Andreas the specifics of what we need and I apologize for that and I'll try to do that now. First, you'll need to clone these two repos if you haven't already: ESMCI CIME: [email protected]:ESMCI/cime.git ACME: [email protected]:ACME-Climate/ACME.git There are a multitude of XML configuration files in both repos. The core problem I'd like you and Andreas to solve is ensuring that XML changes that were made on the ACME get ported correctly to ESMCI CIME. There are 3 areas in particular that I'm concerned about. First, the grid configuration. This file lives in the following locations: In ACME: cime/scripts/Tools/config_grid.xml In ESMCI: cime_config/acme/config_grids.xml This file has three sections (grid list, domains, and grid maps) and the format of each section has completely changed. Grid lists: ACME: <GRID sname="gx1v6_gx1v6" alias="g16_g16" compset="DATM.+DROF" support_level="Non-standard grid for testing of the interpolation in DATM rather than coupler"> a%gx1v6_l%gx1v6_oi%gx1v6_r%rx1_m%gx1v6_g%null_w%null </GRID> ESMCI: <grids> <grid compset="DATM.+DROF"> <lname>a%gx1v6_l%gx1v6_oi%gx1v6_r%rx1_m%gx1v6_g%null_w%null</lname> <sname>gx1v6_gx1v6</sname> <alias>g16_g16</alias> <support>Non-standard grid for testing of the interpolation in DATM rather than coupler</support> </grid> Domains: ACME: <gridhorz name="1x1_numaIA">> <nx>1</nx> <ny>1</ny> <desc>1x1 Numa Iowa -- only valid for DATM/CLM compset</desc> </gridhorz> ... <griddom grid="1x1_numaIA" mask="reg"> <LND_DOMAIN_FILE>domain.lnd.1x1pt-numaIA_navy.110106.nc</LND_DOMAIN_FILE> <LND_DOMAIN_PATH>$DIN_LOC_ROOT/share/domains/domain.clm</LND_DOMAIN_PATH> </griddom> ESMCI: <domains> <domain name="1x1_numaIA"> <nx>1</nx> <ny>1</ny> <file lnd_mask="reg">domain.lnd.1x1pt-numaIA_navy.110106.nc</file> <path lnd_mask="reg">$DIN_LOC_ROOT/share/domains/domain.clm</path> <desc>1x1 Numa Iowa -- only valid for DATM/CLM compset</desc> </domain> This one is especially tricky as you can see on the ACME side, the configs are spread across multiple higher-level elements. Grid Maps: Mercifully, the grip map configuration format is still the same on both side, so you'll only need to check for matches: <gridmap atm_grid="512x1024" lnd_grid="0.23x0.31" > <ATM2LND_FMAPNAME>cpl/gridmaps/T341/map_T341_to_fv0.23x0.31_aave_110413.nc</ATM2LND_FMAPNAME> <ATM2LND_SMAPNAME>cpl/gridmaps/T341/map_T341_to_fv0.23x0.31_aave_110413.nc</ATM2LND_SMAPNAME> <LND2ATM_FMAPNAME>cpl/gridmaps/fv0.23x0.31/map_fv0.23x0.31_to_T341_aave_110413.nc</LND2ATM_FMAPNAME> <LND2ATM_SMAPNAME>cpl/gridmaps/fv0.23x0.31/map_fv0.23x0.31_to_T341_aave_110413.nc</LND2ATM_SMAPNAME> </gridmap> Second, the case configuration entry definitions. This data lives in the following locations: In ACME: cime/scripts/Tools/config_definition.xml In ESMCI: driver_cpl/cime_config/config_component.xml and one per component ACME: <entry id="CASEROOT" type="char" valid_values="" value="UNSET" group="case_def" sdesc="full pathname of case" ></entry> ESMCI: <entry id="CASEROOT"> <type>char</type> <default_value>UNSET</default_value> <group>case_def</group> <file>env_case.xml</file> <desc>full pathname of case</desc> </entry> This one is a little easier. Things that used to be attributes are now subelements. The main issue is that, on the ACME side, ALL entries are defined in this file while on the ESMCI side, each component is expected to define its own entries and only entries that are globally relevant go in this file. Third, the env-mach-specific settings. This file data in the following locations: ACME: cime/machines-acme/env_mach_specific* CIME: cime_config/acme/machines/config_machines.xml ACME (a shell script per machine): module purge module load python/2.7.9 module load gcc/5.1.0/openmpi/1.8.7 module load cmake/2.8.12 module load netcdf/4.3.2/gcc/5.1.0/openmpi/1.8.7 setenv NETCDFROOT $SEMS_NETCDF_ROOT setenv PNETCDFROOT $SEMS_NETCDF_ROOT ESMCI (inside machine XML block): <module_system type="module"> <init_path lang="python">/usr/share/Modules/init/python.py</init_path> <init_path lang="perl">/usr/share/Modules/init/perl.pm</init_path> <init_path lang="sh">/usr/share/Modules/init/sh</init_path> <init_path lang="csh">/usr/share/Modules/init/csh</init_path> <cmd_path lang="python">/usr/bin/modulecmd python</cmd_path> <cmd_path lang="perl">/usr/bin/modulecmd perl</cmd_path> <cmd_path lang="csh">module</cmd_path> <cmd_path lang="sh">module</cmd_path> <modules> <command name="purge"/> <command name="load">python/2.7.9</command> <command name="load">gcc/5.1.0/openmpi/1.8.7</command> <command name="load">cmake/2.8.12</command> <command name="load">netcdf/4.3.2/gcc/5.1.0/openmpi/1.8.7</command> </modules> </module_system> <environment_variables> <env name="NETCDFROOT">$SEMS_NETCDF_ROOT</env> <env name="PNETCDFROOT">$SEMS_NETCDF_ROOT</env> </environment_variables> This will be challenging to automate because ACME/ESMCI are expressing this data in completely different languages. Summary These config issues are the main thing holding up the ACME transition to ESMCI so this stuff is top priority. We need specifics on who intends to do what and within what timeframe. I will help in anyway I can. -Jim From: "Dubey, Anshu" <adubey(a)anl.gov<mailto:[email protected]>> Date: Wednesday, May 25, 2016 at 10:32 AM To: "Foucar, James G" <jgfouca(a)sandia.gov<mailto:[email protected]>> Subject: [EXTERNAL] script to test Jim, The script is already set up to read the files included in the tarball. All you have to do it to run it, you will see that it will go through the first record in the second file, not find a match so move on to the second one. It will find a match there and stop. Comparison is on the text value of each element, and 'acme' and 'cesm' are treated as special cases. If you think this is the basic functionality you are looking for, then we can put appropriate loops around with some tweaking to be done. One other thing I may need to do is to provide more equivalences like 'acme'... If there are likely to be. Best, Anshu

1 0

[Confluence] Jayesh Krishna mentioned you on "Hackathon Ideas for 2016 All-Hands Meeting"
by Jayesh Krishna (Confluence) 02 May '16

02 May '16

1 0