June 2008 - Mg-rast - mailman3.cels.anl.gov

Re: [mg-rast] upload error on mqrast
by Mark D'Souza 30 Jun '08

30 Jun '08

Hi Stuart, The MG-RAST system requires the submitted sequences to be in fasta format either as plain text (*.fna, *.fa, or *.fasta) or as fasta files which have been tarred and gzipped (.tgz). Please redo the submission with one of these formats. My aplogies for the inconvenience, we are very close to releasing version 2.0 for this site which =will= accept untarred gzipped files. Regards, Mark D'Souza -- for the MG-RAST team http://metagenomics.nmpdr.org/ On Mon, 30 Jun 2008 Stuart.Denman(a)csiro.au wrote: > Hi > > I have been trying to upload some 454 data. I have used gzip to compress > the fasta batch file, but I keep getting the following error from your > site "Unfortunately I have not been able to find any fasta files in the > upload." Below is the first three sequences from the file. Can you let > me know if I need to change the format. > > > > >E9ENY9P01A1JLL_length=249_xy=0310_2023_region=1_run=R_2008_04_26_03_00_ > 35_ > > GACCTTGCGCCACCGGTTGGGCGCGCCGAACGTCAGCAGTTTCAACGGCTATTGGTCTGG > > CTGGTTGCTAATGTCTATCCAACATTCACTTTCGCCGATTACCCTGAGCGTTGGGCTCCT > > GACGCACCTGAACAGTTAAAGAAGAATGTAATTGAATATCGTAAATCGCTTTATATCTGG > > CTGAATTCACAATTGACCGCTGAACCTTATGCGTTTGGTGAGCAACTAACGCTGGTGGAC > > TGTTATCTT > > >E9ENY9P01CWBWY_length=57_xy=1071_0960_region=1_run=R_2008_04_26_03_00_3 > 5_ > > ATGCAGAAATCGGCGAGGGAATGTGCAGAAGTGCTGTATATGAAAAGGCATTGGTCG > > >E9ENY9P01A8QCC_length=66_xy=0392_1482_region=1_run=R_2008_04_26_03_00_3 > 5_ > > GGGGGGCCCCCTGTGCTGTGCGGAAAGAGGATTTTTTGCAGGATGGGGGAGGGCAACGTC > > AGGGTC > > Thanks > > Stuart > > > > > > > > > > Stuart Denman > > CSIRO Livestock Industries > 306 Carmody Rd, St Lucia QLD 4067, Australia > Ph: +61-7-3214 2273 Fax: +61-7-3214 2900 > > Stuart.Denman(a)csiro.au > > > > > >

1 0

upload error on mqrast
by Stuart.Denman＠csiro.au 30 Jun '08

30 Jun '08

Hi I have been trying to upload some 454 data. I have used gzip to compress the fasta batch file, but I keep getting the following error from your site "Unfortunately I have not been able to find any fasta files in the upload." Below is the first three sequences from the file. Can you let me know if I need to change the format. >E9ENY9P01A1JLL_length=249_xy=0310_2023_region=1_run=R_2008_04_26_03_00_ 35_ GACCTTGCGCCACCGGTTGGGCGCGCCGAACGTCAGCAGTTTCAACGGCTATTGGTCTGG CTGGTTGCTAATGTCTATCCAACATTCACTTTCGCCGATTACCCTGAGCGTTGGGCTCCT GACGCACCTGAACAGTTAAAGAAGAATGTAATTGAATATCGTAAATCGCTTTATATCTGG CTGAATTCACAATTGACCGCTGAACCTTATGCGTTTGGTGAGCAACTAACGCTGGTGGAC TGTTATCTT >E9ENY9P01CWBWY_length=57_xy=1071_0960_region=1_run=R_2008_04_26_03_00_3 5_ ATGCAGAAATCGGCGAGGGAATGTGCAGAAGTGCTGTATATGAAAAGGCATTGGTCG >E9ENY9P01A8QCC_length=66_xy=0392_1482_region=1_run=R_2008_04_26_03_00_3 5_ GGGGGGCCCCCTGTGCTGTGCGGAAAGAGGATTTTTTGCAGGATGGGGGAGGGCAACGTC AGGGTC Thanks Stuart Stuart Denman CSIRO Livestock Industries 306 Carmody Rd, St Lucia QLD 4067, Australia Ph: +61-7-3214 2273 Fax: +61-7-3214 2900 Stuart.Denman(a)csiro.au

1 0

Olavius endosymbionts metagenomes
by Rob Edwards 29 Jun '08

29 Jun '08

Hey Manuel I was looking at the metagenomics rast and noticed that you submitted a metagenome with large contigs to the service which is taking a very long to process. You should probably try extracting those larger contigs and running them through the rast service - it is much better at annotating complete or nearly complete genomes, and will work great for contigs longer than about 100kb. Rob

1 0

metagenome ftp data
by Mark D'Souza 27 Jun '08

27 Jun '08

Hi Luis, The fragment sequences in the fasta files =are= the contigs used by MG-RAST as submitted to the site, no attempt is made to assemble them. All the analysis available at the MG-RAST site is based on the fragments. Regards, Mark D'Souza -- for the MG-RAST team Date: Mon, 23 Jun 2008 19:03:14 -0500 (CDT) From: llozano(a)ccg.unam.mx To: info(a)theseed.org Subject: [info] metagenomes data... Hi, I was looking for some metagenomic data in your server. I need to obtain the contigs for some metagenomic projects. With the SEED viewer I notice that the metagenomic projects have "size information" in bps and "number of fragments". I believe that the "number of fragments" refers to the number of contigs for that metagenome project. When I search in your FTP site I found that in: ftp://ftp.theseed.org/metagenomes/contigs I could download the metagenomic archives *.fa.gz. I look at them and found that they don't seem to be contigs, the look like the 454 simple readings. Do you have some archives with the contigs of the metagenome projects? Thanks. Luis Lozano. Genomic Science Center. UNAM Cuernavaca, Morelos, Mexico.

1 0

Re: [mg-rast] on the topic of memory use
by Tobias Paczian 26 Jun '08

26 Jun '08

Hmmmm... I will dive into this one this afternoon... Cheers Tobi On Jun 26, 2008, at 12:07 PM, Mark D'Souza wrote: > I have seen that before, and if I remember correctly it had nothing > to do > with the PPO, and vanished when I restarted my browser. > > and it did again! > > we are still not out of the woods. I assume the problem occurs when > the > same user opens multiple pages, and the CMORE class will be using a > single > user -- unless we cannot fix this. > > Anyone have any ideas why the user management code would behave > like this? > > Mark > > On Thu, 26 Jun 2008, Robert Olson wrote: > >> in the mg-rast error log: >> >> [Thu Jun 26 11:26:56 2008] [error] [client 140.221.34.105] Deep >> recursion on subroutine "Tie::Array::SPLICE" at >> /vol/mg-rast-prod/FIGdisk/dist/releases/dev/common/lib/PPO/ >> DBSQLArray.pm line 105., referer: http://mg-rast.mcs. >> anl.gov/mg-rast/FIG/rast.cgi?page=JobDetails&job=834 >> >> >

1 0

Re: [mg-rast] on the topic of memory use
by Mark D'Souza 26 Jun '08

26 Jun '08

I have seen that before, and if I remember correctly it had nothing to do with the PPO, and vanished when I restarted my browser. and it did again! we are still not out of the woods. I assume the problem occurs when the same user opens multiple pages, and the CMORE class will be using a single user -- unless we cannot fix this. Anyone have any ideas why the user management code would behave like this? Mark On Thu, 26 Jun 2008, Robert Olson wrote: > in the mg-rast error log: > > [Thu Jun 26 11:26:56 2008] [error] [client 140.221.34.105] Deep > recursion on subroutine "Tie::Array::SPLICE" at > /vol/mg-rast-prod/FIGdisk/dist/releases/dev/common/lib/PPO/ > DBSQLArray.pm line 105., referer: http://mg-rast.mcs. > anl.gov/mg-rast/FIG/rast.cgi?page=JobDetails&job=834 > >

1 0

on the topic of memory use
by Robert Olson 26 Jun '08

26 Jun '08

in the mg-rast error log: [Thu Jun 26 11:26:56 2008] [error] [client 140.221.34.105] Deep recursion on subroutine "Tie::Array::SPLICE" at /vol/mg-rast-prod/FIGdisk/dist/releases/dev/common/lib/PPO/ DBSQLArray.pm line 105., referer: http://mg-rast.mcs. anl.gov/mg-rast/FIG/rast.cgi?page=JobDetails&job=834

1 0

Re: [mg-rast] Fwd: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN **
by Robert Olson 26 Jun '08

26 Jun '08

The 4G CGI mem-use limit is in place now. I have also changed the apache config to run as mgrastprod instead of apache. 'twas an oversight that it wasn't already like that. --bob On Jun 26, 2008, at 11:23 AM, Robert Olson wrote: > Ganglia notes the following signature before the hang > > <Picture 31.png> > > note the large incoming traffic spikes that go along with the > memory use. This is consistent with a web interface page that is > ingesting a large amount of data to display, like the GenomeBrowser > does. > > Right now I see this: > > 2308 apache 15 0 3492m 2.8g 1796 R 26 17.8 4:00.82 perl > 1334 apache 18 0 4663m 3.3g 1080 D 0 20.8 6:28.44 perl > > Looking at a ps with environment variable display, I see each is > executing > > /mg-rast/FIG/metagenomics.cgi? > page=MetagenomeOverview&metagenome=4440977.3 > > so someone should have a look at that to see what it's doing. > > In the meantime I'm thinking I'm going to set a memory-use limit in > the apache config to 4G or so. None of our production CGIs should > be using that much. > > On Jun 26, 2008, at 11:07 AM, Mark D'Souza wrote: > >> I had some pages open, and opened a number of overview pages >> simultaneously, which may have caused the spike. >> >> This is something the machine should be able to handle, when it >> comes back >> I will run some tests with the pages I was opening. >> >> The CMORE class runs next week, starting the 30th, and we will be >> having >> up to 20 students browsing the site, so we should really get this >> fixed. >> >> Mark >> >> >> On Thu, 26 Jun 2008, Robert Olson wrote: >> >>> mg-rast memory use spiked again just now; anyone happen to notice >>> what was going on? I'm guessing it's going to be hung for a bit. >>> >>> >>> Begin forwarded message: >>> >>>> From: nagios(a)mikey.mcs.anl.gov >>>> Date: June 26, 2008 10:53:47 AM CDT >>>> To: olson(a)mcs.anl.gov >>>> Subject: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN ** >>>> >>>> ***** Nagios ***** >>>> >>>> Notification Type: PROBLEM >>>> >>>> Service: check free memory >>>> Host: mg-rast >>>> Address: 192.5.200.102 >>>> State: UNKNOWN >>>> >>>> Date/Time: Thu Jun 26 10:53:47 CDT 2008 >>>> >>>> Additional Info: >>>> >>>> SNMP problem - No data received from host >>> >>> >

1 0

Re: [mg-rast] Fwd: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN **
by Robert Olson 26 Jun '08

26 Jun '08

Ganglia notes the following signature before the hang note the large incoming traffic spikes that go along with the memory use. This is consistent with a web interface page that is ingesting a large amount of data to display, like the GenomeBrowser does. Right now I see this: 2308 apache 15 0 3492m 2.8g 1796 R 26 17.8 4:00.82 perl 1334 apache 18 0 4663m 3.3g 1080 D 0 20.8 6:28.44 perl Looking at a ps with environment variable display, I see each is executing /mg-rast/FIG/metagenomics.cgi? page=MetagenomeOverview&metagenome=4440977.3 so someone should have a look at that to see what it's doing. In the meantime I'm thinking I'm going to set a memory-use limit in the apache config to 4G or so. None of our production CGIs should be using that much. On Jun 26, 2008, at 11:07 AM, Mark D'Souza wrote: > I had some pages open, and opened a number of overview pages > simultaneously, which may have caused the spike. > > This is something the machine should be able to handle, when it > comes back > I will run some tests with the pages I was opening. > > The CMORE class runs next week, starting the 30th, and we will be > having > up to 20 students browsing the site, so we should really get this > fixed. > > Mark > > > On Thu, 26 Jun 2008, Robert Olson wrote: > >> mg-rast memory use spiked again just now; anyone happen to notice >> what was going on? I'm guessing it's going to be hung for a bit. >> >> >> Begin forwarded message: >> >>> From: nagios(a)mikey.mcs.anl.gov >>> Date: June 26, 2008 10:53:47 AM CDT >>> To: olson(a)mcs.anl.gov >>> Subject: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN ** >>> >>> ***** Nagios ***** >>> >>> Notification Type: PROBLEM >>> >>> Service: check free memory >>> Host: mg-rast >>> Address: 192.5.200.102 >>> State: UNKNOWN >>> >>> Date/Time: Thu Jun 26 10:53:47 CDT 2008 >>> >>> Additional Info: >>> >>> SNMP problem - No data received from host >> >>

1 0

Re: [mg-rast] Fwd: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN **
by Mark D'Souza 26 Jun '08

26 Jun '08

I had some pages open, and opened a number of overview pages simultaneously, which may have caused the spike. This is something the machine should be able to handle, when it comes back I will run some tests with the pages I was opening. The CMORE class runs next week, starting the 30th, and we will be having up to 20 students browsing the site, so we should really get this fixed. Mark On Thu, 26 Jun 2008, Robert Olson wrote: > mg-rast memory use spiked again just now; anyone happen to notice > what was going on? I'm guessing it's going to be hung for a bit. > > > Begin forwarded message: > > > From: nagios(a)mikey.mcs.anl.gov > > Date: June 26, 2008 10:53:47 AM CDT > > To: olson(a)mcs.anl.gov > > Subject: ** PROBLEM alert - mg-rast/check free memory is UNKNOWN ** > > > > ***** Nagios ***** > > > > Notification Type: PROBLEM > > > > Service: check free memory > > Host: mg-rast > > Address: 192.5.200.102 > > State: UNKNOWN > > > > Date/Time: Thu Jun 26 10:53:47 CDT 2008 > > > > Additional Info: > > > > SNMP problem - No data received from host > >

1 0