June 2008 - Mg-rast - mailman3.cels.anl.gov

Fwd: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN **
by Robert Olson 26 Jun '08

26 Jun '08

mg-rast memory use spiked again just now; anyone happen to notice what was going on? I'm guessing it's going to be hung for a bit. Begin forwarded message: > From: nagios(a)mikey.mcs.anl.gov > Date: June 26, 2008 10:53:47 AM CDT > To: olson(a)mcs.anl.gov > Subject: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN ** > > ***** Nagios ***** > > Notification Type: PROBLEM > > Service: check free memory > Host: mg-rast > Address: 192.5.200.102 > State: UNKNOWN > > Date/Time: Thu Jun 26 10:53:47 CDT 2008 > > Additional Info: > > SNMP problem - No data received from host

1 0

Re: [mg-rast] queue
by Rob Edwards 26 Jun '08

26 Jun '08

Figured as much .... everything is just stalled whenever the last active stage completes. No new stages are run. Luckily we have quite a few jobs in the queue, so things haven't been idle :) Rob On Jun 26, 2008, at 7:35 AM, Robert Olson wrote: > hm. I've been tinkering with the pipeline to make the old & new > servers > coexist; perhaps there is a problem. are no new jobs going thru? > > On Thu, 26 Jun 2008, Rob Edwards wrote: > >> Is there a reason that the mg-rast queue has been suspended? Sims >> jobs >> are still chunking through, but nothing new is being done. >> >> Rob >> >>

1 0

Re: [mg-rast] queue
by Robert Olson 26 Jun '08

26 Jun '08

hm. I've been tinkering with the pipeline to make the old & new servers coexist; perhaps there is a problem. are no new jobs going thru? On Thu, 26 Jun 2008, Rob Edwards wrote: > Is there a reason that the mg-rast queue has been suspended? Sims jobs > are still chunking through, but nothing new is being done. > > Rob > >

1 0

queue
by Rob Edwards 26 Jun '08

26 Jun '08

Is there a reason that the mg-rast queue has been suspended? Sims jobs are still chunking through, but nothing new is being done. Rob

1 0

Re: [mg-rast] Procedure for updating production server II
by Andreas Wilke 25 Jun '08

25 Jun '08

I run into a problem with ssh: Installing the public key on one of the MCS systems, ensure that there are no line breaks in the appended public key in .ssh/authorized_key Preliminary setup: -2. Create a ssh keypair that is owned by user rastcode but with a passphrase that you know. As user rastcode on mg-rast.mcs.anl.gov run: ssh-keygen -t dsa When it asks for a pathname for the keys, enter /home/rastcode/.ssh/ your-username The public key is then in /home/rastcode/.ssh/your-username.pub. Append the contents of that file to the .ssh/authorized_keys file in your home directory on one of the MCS systems (terra/shakey/etc). Ensure that there are no line breaks in the copied public key. -1. Log into mg-rast, sudo to rastcode. Then start a ssh-agent and add the private key: ssh-agent bash ssh-add /home/rastcode/.ssh/your-username The model we use is that a particular person (the release master for the release) owns the checkout on the test server. There needs to be a real checkout (not an anonymous one) because the running code release needs to be tagged. The procedure from the start of a new test release cycle: Release owner checks out a new copy of mg-rast code. As user rastcode, with an ssh-agent in place as above: export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-test/FIGdisk source config/fig-user-env.sh cd dist/releases Here, you need to choose a release name. I've been using the server name & date mkdir mgrast_2008_0625 cd mgrast_2008_0625 cvs co mgrast cp ReleaseTools/Makefile.top Makefile New code is checked out. To activate it, use switch_to_release: switch_to_release mgrast_2008_0625 The new checkout is now active. Test the server. Make sure the pages work. Run some genomes through. Make sure they worked. When it is decided that this release is ready to go, we will tag it in CVS so we can ensure the production server is running exactly the same version of the code. export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-test/FIGdisk source config/fig-user-env.sh cd dist/releases/mgrast_2008_0625 cvs tag -r mgrast_rel_2008_0625 Now all the MGRAST code files corresponding to the running test server are tagged as mgrast_rel_2008_0625. We need to create the new production server. Bring up a new login to mg-rast, sudo to rastcode. This process is similar to the initial test server setup, but checks out the tagged version instead of the latest version (unrelated code may have been updated since the test cycle started - we do not want those changes). export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-prod/FIGdisk source config/fig-user-env.sh cd dist/releases mkdir mgrast_2008_0625 cvs export -r mgrast_rel_2008_0625 mgrast cp ReleaseTools/Makefile.top Makefile Then to make it active: switch_to_release mgrast_2008_0625 ------------------------------------------------------ Andreas Wilke Argonne National Laboratory Mathematics and Computer Science Division Bldg. 221, Room A147 9700 South Cass Avenue Argonne, IL 60439 (630) 252-3190 (phone) (630) 252-5986.(fax) -------------------------------------------------------

1 0

Re: [mg-rast] missing tables?
by Mark D'Souza 25 Jun '08

25 Jun '08

Bob, We need 3 metagenomes to have fully functional analyses for the CMORE class next week, so I will go ahead and do the port as below. After that can I run this for every genome in mg-rast-prod/jobs/ ? Mark On Wed, 25 Jun 2008, Robert Olson wrote: > those are genomes that have not been loaded yet. You can trigger a > load by sourcing the correct fig-user-env and running > > mg_port_job /vol/mg-rast-test/jobs/565 > > (for instance, for the job below). > > --bob > > On Jun 25, 2008, at 11:05 AM, Mark D'Souza wrote: > > > Hi, > > > > From the v2.0 production site, the metabolic reconstruction and > > phylogenetic classification produce the error: > > > > DBD::Pg::st execute failed: ERROR: relation "tax_sim_best_by_psc_565" > > does not exist > > > > URLs: > > http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi? > > page=MetagenomeProfile&dataset=SEED:subsystem_tax&metagenome=4440690.3 > > http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi? > > page=MetagenomeProfile&dataset=rdp:16s_taxonomy&metagenome=4440690.3 > > > > I tried a couple of genomes and get the same error. > > > > Mark > > > >

1 0

Re: [mg-rast] missing tables?
by Robert Olson 25 Jun '08

25 Jun '08

those are genomes that have not been loaded yet. You can trigger a load by sourcing the correct fig-user-env and running mg_port_job /vol/mg-rast-test/jobs/565 (for instance, for the job below). --bob On Jun 25, 2008, at 11:05 AM, Mark D'Souza wrote: > Hi, > > From the v2.0 production site, the metabolic reconstruction and > phylogenetic classification produce the error: > > DBD::Pg::st execute failed: ERROR: relation "tax_sim_best_by_psc_565" > does not exist > > URLs: > http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi? > page=MetagenomeProfile&dataset=SEED:subsystem_tax&metagenome=4440690.3 > http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi? > page=MetagenomeProfile&dataset=rdp:16s_taxonomy&metagenome=4440690.3 > > I tried a couple of genomes and get the same error. > > Mark >

1 0

missing tables?
by Mark D'Souza 25 Jun '08

25 Jun '08

Hi, >From the v2.0 production site, the metabolic reconstruction and phylogenetic classification produce the error: DBD::Pg::st execute failed: ERROR: relation "tax_sim_best_by_psc_565" does not exist URLs: http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi?page=MetagenomeProf… http://mg-rast.mcs.anl.gov/mg-rast/FIG/metagenomics.cgi?page=MetagenomeProf… I tried a couple of genomes and get the same error. Mark

1 0

Procedure for updating production server
by Robert Olson 25 Jun '08

25 Jun '08

Preliminary setup: -2. Create a ssh keypair that is owned by user rastcode but with a passphrase that you know. As user rastcode on mg-rast.mcs.anl.gov run: ssh-keygen -t dsa When it asks for a pathname for the keys, enter /home/rastcode/.ssh/ your-username The public key is then in /home/rastcode/.ssh/your-username.pub. Append the contents of that file to the .ssh/authorized_keys file in your home directory on one of the MCS systems (terra/shakey/etc). -1. Log into mg-rast, sudo to rastcode. Then start a ssh-agent and add the private key: ssh-agent bash ssh-add /home/rastcode/.ssh/your-username The model we use is that a particular person (the release master for the release) owns the checkout on the test server. There needs to be a real checkout (not an anonymous one) because the running code release needs to be tagged. The procedure from the start of a new test release cycle: Release owner checks out a new copy of mg-rast code. As user rastcode, with an ssh-agent in place as above: export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-test/FIGdisk source config/fig-user-env.sh cd dist/releases Here, you need to choose a release name. I've been using the server name & date mkdir mgrast_2008_0625 cd mgrast_2008_0625 cvs co mgrast cp ReleaseTools/Makefile.top Makefile New code is checked out. To activate it, use switch_to_release: switch_to_release mgrast_2008_0625 The new checkout is now active. Test the server. Make sure the pages work. Run some genomes through. Make sure they worked. When it is decided that this release is ready to go, we will tag it in CVS so we can ensure the production server is running exactly the same version of the code. export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-test/FIGdisk source config/fig-user-env.sh cd dist/releases/mgrast_2008_0625 cvs tag -r mgrast_rel_2008_0625 Now all the MGRAST code files corresponding to the running test server are tagged as mgrast_rel_2008_0625. We need to create the new production server. Bring up a new login to mg-rast, sudo to rastcode. This process is similar to the initial test server setup, but checks out the tagged version instead of the latest version (unrelated code may have been updated since the test cycle started - we do not want those changes). export CVS_RSH=ssh export CVSROOT=:ext:[email protected]:/disks/cvs/bio cd /vol/mg-rast-prod/FIGdisk source config/fig-user-env.sh cd dist/releases mkdir mgrast_2008_0625 cvs export -r mgrast_rel_2008_0625 mgrast cp ReleaseTools/Makefile.top Makefile Then to make it active: switch_to_release mgrast_2008_0625

1 0

addition to the metagenome page
by Folker Meyer 24 Jun '08

24 Jun '08

Hi, as discussed, here is a text for the metagenome overview page (note it includes some variables). Folker PS: Liz and I wrote the text and were pretty tired. There might be the need for some changes.

1 0