October 2009 - Mg-rast - mailman3.cels.anl.gov

Running a large batch job
by Hazem Sharaf 30 Oct '09

30 Oct '09

Dear Support, I have a metagenome sample sequenced using pyrosequencing which resulted in a large dataset that has 1.2 million reads. I have read the guidelines for uploading on the website which states that files over 30MB should be compressed into a .tgz archive. However, after compressing the data, the resulted archive is about 170MB. Should I be able to upload this file to your server and have it running through your system? -- Regards, Hazem Sharaf Computer Specialist Biology Department American University in Cairo

1 0

Re: [mg-rast] MG-RAST JOBS question
by dsouza＠mcs.anl.gov 30 Oct '09

30 Oct '09

Hi Ami, 4926 is computing right now actually. It is about 2/3 through and should get done today, 4927 should be done in a few days. 4977 will take longer, maybe a week. ----- "Ami M. Smith" <smitham(a)wvstateu.edu> wrote: > From: "Ami M. Smith" <smitham(a)wvstateu.edu> > To: mg-rast(a)mcs.anl.gov > Sent: Friday, October 30, 2009 12:08:38 PM GMT -06:00 US/Canada Central > Subject: [mg-rast] MG-RAST JOBS question > > Hello! > > I have three jobs (4977, 4927, 4926) submitted to MG-RAST two of which > were submitted 1 month ago. I was wondering if it is possible to get > an estimated time of completion for these jobs. The reason I ask is > that we have decided that for jobs 4926 and 4927 we are going to do > some additional preproccessing and then resubmit them. If completion > of the current analysis will not occur in the next week or so then it > may be best to delete these jobs at which point I would resubmit them > after preproccessing. If they will be completed soon then we will like > to leave them in the queue. > > Thank you! > > > Ami M. Smith > Postdoctoral Scientist > Environmental Microbiology > West Virginia State University > Hamblin Hall Room 208B > Institute, WV 25112-1000 > smitham(a)wvstateu.edu > office: 304-766-5168 > fax: 304-766-4199 -- Regards, Mark D'Souza -- for the MG-RAST team mg-rast(a)mcs.anl.gov http://metagenomics.nmpdr.org/

1 0

MG-RAST JOBS question
by Ami M. Smith 30 Oct '09

30 Oct '09

Hello! I have three jobs (4977, 4927, 4926) submitted to MG-RAST two of which were submitted 1 month ago. I was wondering if it is possible to get an estimated time of completion for these jobs. The reason I ask is that we have decided that for jobs 4926 and 4927 we are going to do some additional preproccessing and then resubmit them. If completion of the current analysis will not occur in the next week or so then it may be best to delete these jobs at which point I would resubmit them after preproccessing. If they will be completed soon then we will like to leave them in the queue. Thank you! Ami M. Smith Postdoctoral Scientist Environmental Microbiology West Virginia State University Hamblin Hall Room 208B Institute, WV 25112-1000 smitham(a)wvstateu.edu office: 304-766-5168 fax: 304-766-4199

1 0

MG-RAST problem?
by Rachel Poretsky 29 Oct '09

29 Oct '09

Hi, When using the KEGG map tool, I am unable to get a list of the sequences that are attributed to a particular enzyme in a pathway. They show up when I mouse over the EC number, but vanish when I click on it. Instead I get to a blank table (with headings only) and an option to export table (which does nothing). Is there a problem on my end or does this tool still not work? Thanks, Rachel -- Rachel Poretsky Postdoctoral Scholar Caltech- Geological and Planetary Sciences MC 100-23 Pasadena, CA 91125 626-395-2536 (office) 626-395-6018 (lab)

1 0

Jobs
by Thibault Varin 27 Oct '09

27 Oct '09

Hello, I was wondering if you could tell me when my jobs (5039 and 5042) will be completed ? Thank you very much, Thibault Varin PhD student Laval University.

1 0

Re: [mg-rast] question on MG-RAST
by Gary Xie 26 Oct '09

26 Oct '09

Hi, Mark, Would you tell me more info about how MG-rast generate that tiny table (on the right) of metagenome overview page. Is it based on the top blast hit. If so, what kind of parameter has been used. I am assuming those "other" in this table should be equal to the "non- classified" number in the "sequence profile" page, when the SEED and RDP are chosen as the dataset. But instead, I have noticed the numbers between two are quite different in the following example: 9% in the "overview page" vs 50% in the "sequence profile" page when SEED is the dataset. Would you explain on this? THanks for your time. Gary http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeOverview&meta… On Oct 25, 2009, at 2:34 PM, dsouza(a)mcs.anl.gov wrote: > Hi Gary, > > Regarding the part of your question (1) I did not address ... >>> Another table on the metagenome overview page is showing taxonomic >>> distribution based on best protein similarity to SEED and 16S based >>> similarity to RDP. But i have noticed that significant amount (about >>> 10%) has been assigned to "other". Would you explain what are those >>> "other" > > The number under 'other' is the number of sequences which did not > get classified under the other categories, i.e. it is just the > unclassified sequences. > > Mark >

1 0

Re: [mg-rast] question on MG-RAST
by Gary Xie 26 Oct '09

26 Oct '09

HI, Mark, Thanks for responding all my question. I really appreciated. I have following three further questions. I know you guys are busying preparing for this week soil workshop, if you had a chance to address, that would be great. Otherwise, our LANL metagenome team lead--Patrick Chain will ask these questions at your mg-rast workshop on behalf of me. Thanks for your time. Gary = = ======================================================================== For my previous question 1, Since your QC step includes removing all duplicated sequences, is this going to jeopardize your heatmap? For example, if you have the shotgun metagenome unassembled reads with 1x or less coverage, i guess this QC step is going to mask the true abundance profile reflected in the heatmap, unless you have the mechanism to bring back the removed number when generating the heat map. Of course, the pre-sequencing rRNA survey should have already given your some ideas of the composition of your environment. For my question 2, Since what you classified are "reads", I am assuming it is based on the top blast. And I know your SEED database doesn't include those environmental sequences, If you choose the phylogenetic profile dataset--SEED, I am afraid you are going to overestimate the number of reads that can be phylogenetic classified, because likely your top hits are from those known genomes in SEED database. Is that right? For my question 4, Does SEED include all finished/unfinished Eukaryotic genomes from Genbank? On Oct 25, 2009, at 2:39 AM, dsouza(a)mcs.anl.gov wrote: > Hi Gary, > > I am going to work through some of your questions: > 1. There are two possible sources for the discrepancy in the number > of sequences, when the job is submitted the user can instruct the > system to remove duplicate sequences, we also drop sequences less > than 10bp long. One or both of these could contribute to the > difference in the sequence numbers. > > 2. 'Classified' on this page refers to phylogenetic classification, > not functional . A hit to a hypothetical protein is therefore > labelled as classified as the SEED genome of the hit is being used > to classify the metagenomic sequence. > > 4. This is probably reflecting the composition of the respective > databases, I know the SEED protein sequence is biased towards > prokaryotic sequences. > > Let me know if you want further clarification. > > Regards, > Mark > > ----- "Gary Xie" <xie(a)lanl.gov> wrote: > >> From: "Gary Xie" <xie(a)lanl.gov> >> To: "Gary Xie" <xie(a)lanl.gov> >> Cc: mg-rast(a)mcs.anl.gov, "Rob Edwards" <redwards(a)mcs.anl.gov>, >> "Mark D'Souza" <dsouza(a)mcs.anl.gov>, "Folker Meyer" >> <folker(a)anl.gov> >> Sent: Tuesday, October 20, 2009 12:27:19 PM GMT -06:00 US/Canada >> Central >> Subject: Re: question on MG-RAST >> >> Hi, there, >> I have few questions about the MG-RAST result. Maybe you can explain >> >> to me. >> 1) On the metagenome overview page, there are two summary tables: one >> >> has the "Total no. of sequences" filed, which is the total number of >> >> sequences submitted by the user for this metagenome. I have noticed >> its number is slightly less than "Number of uploaded sequences:" on >> the "job detail" page. I am assuming that MG RAST did some QC >> process >> >> to remove some junk sequences. If so, what are the criterial used >> during this QC process. >> >> 2) On the "sequence profile" page, I have noticed that there is a >> summary graph of the number of sequences that were classified given >> the parameters chosen. If I choose "profile type" as phylogenetic >> profile and dataset as SEED, the sequence number is >> >> "reads" number or "ORF" number??? If the read or ORF matches to a >> hypothetical protein in SEED, is it going to be assigned as >> classified sequence or non-classified?? in other words, are those >> hypothetical proteins considered as "non-classified" in SEED? >> >> >> 4) I am very interested in the soil community profiling, especially >> for Eukaryota distribution. Your phylogenetic profile feature >> provides >> >> user to access these info via a) SEED b) Silva ssu c) Silva lsu >> fields, but I have noticed results are quite different when using >> SEED >> >> vs Silva phylogenetic profile. >> >> http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeProfile&datas… >> >> For example in about sample, if I use the max e-value 1e-10 as the >> cut- >> off, SEED give me only 1.68% as Eukaryota in the pie chart, but both >> >> Silva ssu and lsu give me about 16% as Eukaryota, which is 10 fold >> difference. I certainly understand that rRNA read just a small >> portion >> >> of this metagenome shutgun reads. But this 10 fold different is hard >> >> to explain. >> >> I checked the MG-RAST user guide, I didn't find the answer. >> Thanks >> Gary

1 0

MG RAST hangs
by Jostein Johansen 26 Oct '09

26 Oct '09

Dear MGRAST team, I am experiencing a problem with job 5058 on MGRAST. It has been stuck in this position for for about 5-6 days. Please let me know if there is any problem regards, -- Jostein Johansen FUGE Bioinformatics Platform Helpdesk Programmer, Bioinformatics & Gene Regulation Department of Cancer Research and Molecular Medicine Laboratory Centre,5th floor. Erling Skjalgssons gt. 1 Norwegian University of Science and Technology, NTNU DMF, 7006 Trondheim, Norway Telephone: +47 725 73359

1 0

Re: [mg-rast] question on MG-RAST
by dsouza＠mcs.anl.gov 25 Oct '09

25 Oct '09

Hi Gary, Regarding the part of your question (1) I did not address ... > > Another table on the metagenome overview page is showing taxonomic > > distribution based on best protein similarity to SEED and 16S based > > similarity to RDP. But i have noticed that significant amount (about > > 10%) has been assigned to "other". Would you explain what are those > > "other" The number under 'other' is the number of sequences which did not get classified under the other categories, i.e. it is just the unclassified sequences. Mark ----- dsouza(a)mcs.anl.gov wrote: > From: dsouza(a)mcs.anl.gov > To: "Gary Xie" <xie(a)lanl.gov> > Cc: mg-rast(a)mcs.anl.gov, "Rob Edwards" <redwards(a)mcs.anl.gov>, "Mark D'Souza" <dsouza(a)mcs.anl.gov>, "Folker Meyer" > <folker(a)anl.gov> > Sent: Sunday, October 25, 2009 3:39:06 AM GMT -06:00 US/Canada Central > Subject: Re: question on MG-RAST > > Hi Gary, > > I am going to work through some of your questions: > 1. There are two possible sources for the discrepancy in the number of > sequences, when the job is submitted the user can instruct the system > to remove duplicate sequences, we also drop sequences less than 10bp > long. One or both of these could contribute to the difference in the > sequence numbers. > > 2. 'Classified' on this page refers to phylogenetic classification, > not functional . A hit to a hypothetical protein is therefore labelled > as classified as the SEED genome of the hit is being used to classify > the metagenomic sequence. > > 4. This is probably reflecting the composition of the respective > databases, I know the SEED protein sequence is biased towards > prokaryotic sequences. > > Let me know if you want further clarification. > > Regards, > Mark > > ----- "Gary Xie" <xie(a)lanl.gov> wrote: > > > From: "Gary Xie" <xie(a)lanl.gov> > > To: "Gary Xie" <xie(a)lanl.gov> > > Cc: mg-rast(a)mcs.anl.gov, "Rob Edwards" <redwards(a)mcs.anl.gov>, "Mark > D'Souza" <dsouza(a)mcs.anl.gov>, "Folker Meyer" > > <folker(a)anl.gov> > > Sent: Tuesday, October 20, 2009 12:27:19 PM GMT -06:00 US/Canada > Central > > Subject: Re: question on MG-RAST > > > > Hi, there, > > I have few questions about the MG-RAST result. Maybe you can explain > > > > > to me. > > 1) On the metagenome overview page, there are two summary tables: > one > > > > has the "Total no. of sequences" filed, which is the total number of > > > > > sequences submitted by the user for this metagenome. I have noticed > > > its number is slightly less than "Number of uploaded sequences:" on > > > the "job detail" page. I am assuming that MG RAST did some QC > process > > > > to remove some junk sequences. If so, what are the criterial used > > during this QC process. > > Another table on the metagenome overview page is showing taxonomic > > > distribution based on best protein similarity to SEED and 16S based > > > similarity to RDP. But i have noticed that significant amount (about > > > > > 10%) has been assigned to "other". Would you explain what are those > > > "other" > > > http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeOverview&meta… > > > > 2) On the "sequence profile" page, I have noticed that there is a > > > summary graph of the number of sequences that were classified given > > > the parameters chosen. If I choose "profile type" as phylogenetic > > profile and dataset as SEED, The sequence number in following graph > is > > > > "reads" number or "ORF" number??? If the read or ORF matches to a > > hypothetical protein in SEED, is it going to be assigned as > > classified sequence or non-classified?? in other words, are those > > hypothetical proteins considered as "non-classified" in SEED? > > > > 3) On your job details page, you have two fields --"Number of > uploaded > > > > sequences:" and "Number of sequences:". I have noticed in many case, > > > > > the number of uploaded seq is larger than number of seq. Why they > are > > > > difference? I am assuming that you guys did a QC step, would you let > > > > > me know these criterial? > > > > 4) I am very interested in the soil community profiling, especially > > > for Eukaryota distribution. Your phylogenetic profile feature > provides > > > > user to access these info via a) SEED b) Silva ssu c) Silva lsu > > fields, but I have noticed results are quite different when using > SEED > > > > vs Silva phylogenetic profile. > > > > > http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeProfile&datas… > > > > For example in about sample, if I use the max e-value 1e-10 as the > > cut- > > off, SEED give me only 1.68% as Eukaryota in the pie chart, but both > > > > > Silva ssu and lsu give me about 16% as Eukaryota, which 10 fold > > different. I certainly understand that rRNA read just a small > portion > > > > of this metagenome shutgun reads. But this 10 fold different is hard > > > > > to explain. > > > > I checked the MG-RAST user guide, I didn't find the answer. Do you > > > have any explanation on this? Do you have any recommendation on the > > > parameter (cut-off) I should use? > > Thanks > > Gary

1 0

Re: [mg-rast] question on MG-RAST
by dsouza＠mcs.anl.gov 25 Oct '09

25 Oct '09

Hi Gary, I am going to work through some of your questions: 1. There are two possible sources for the discrepancy in the number of sequences, when the job is submitted the user can instruct the system to remove duplicate sequences, we also drop sequences less than 10bp long. One or both of these could contribute to the difference in the sequence numbers. 2. 'Classified' on this page refers to phylogenetic classification, not functional . A hit to a hypothetical protein is therefore labelled as classified as the SEED genome of the hit is being used to classify the metagenomic sequence. 4. This is probably reflecting the composition of the respective databases, I know the SEED protein sequence is biased towards prokaryotic sequences. Let me know if you want further clarification. Regards, Mark ----- "Gary Xie" <xie(a)lanl.gov> wrote: > From: "Gary Xie" <xie(a)lanl.gov> > To: "Gary Xie" <xie(a)lanl.gov> > Cc: mg-rast(a)mcs.anl.gov, "Rob Edwards" <redwards(a)mcs.anl.gov>, "Mark D'Souza" <dsouza(a)mcs.anl.gov>, "Folker Meyer" > <folker(a)anl.gov> > Sent: Tuesday, October 20, 2009 12:27:19 PM GMT -06:00 US/Canada Central > Subject: Re: question on MG-RAST > > Hi, there, > I have few questions about the MG-RAST result. Maybe you can explain > > to me. > 1) On the metagenome overview page, there are two summary tables: one > > has the "Total no. of sequences" filed, which is the total number of > > sequences submitted by the user for this metagenome. I have noticed > its number is slightly less than "Number of uploaded sequences:" on > the "job detail" page. I am assuming that MG RAST did some QC process > > to remove some junk sequences. If so, what are the criterial used > during this QC process. > Another table on the metagenome overview page is showing taxonomic > distribution based on best protein similarity to SEED and 16S based > similarity to RDP. But i have noticed that significant amount (about > > 10%) has been assigned to "other". Would you explain what are those > "other" > http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeOverview&meta… > > 2) On the "sequence profile" page, I have noticed that there is a > summary graph of the number of sequences that were classified given > the parameters chosen. If I choose "profile type" as phylogenetic > profile and dataset as SEED, The sequence number in following graph is > > "reads" number or "ORF" number??? If the read or ORF matches to a > hypothetical protein in SEED, is it going to be assigned as > classified sequence or non-classified?? in other words, are those > hypothetical proteins considered as "non-classified" in SEED? > > 3) On your job details page, you have two fields --"Number of uploaded > > sequences:" and "Number of sequences:". I have noticed in many case, > > the number of uploaded seq is larger than number of seq. Why they are > > difference? I am assuming that you guys did a QC step, would you let > > me know these criterial? > > 4) I am very interested in the soil community profiling, especially > for Eukaryota distribution. Your phylogenetic profile feature provides > > user to access these info via a) SEED b) Silva ssu c) Silva lsu > fields, but I have noticed results are quite different when using SEED > > vs Silva phylogenetic profile. > > http://metagenomics.nmpdr.org/metagenomics.cgi?page=MetagenomeProfile&datas… > > For example in about sample, if I use the max e-value 1e-10 as the > cut- > off, SEED give me only 1.68% as Eukaryota in the pie chart, but both > > Silva ssu and lsu give me about 16% as Eukaryota, which 10 fold > different. I certainly understand that rRNA read just a small portion > > of this metagenome shutgun reads. But this 10 fold different is hard > > to explain. > > I checked the MG-RAST user guide, I didn't find the answer. Do you > have any explanation on this? Do you have any recommendation on the > parameter (cut-off) I should use? > Thanks > Gary

1 0