July 2011 - Mg-rast - mailman3.cels.anl.gov

Fwd: Data selection errors
by Mark D'Souza 25 Jul '11

25 Jul '11

Hi Travis, This user is having trouble getting the GenBank organism classification for his jobs through the website because of the size of the jobs, how difficult is it to generate the file from the command line and make it available on an ftp site? Regards, Mark Begin forwarded message: > From: anthony bertagnolli <bertagna(a)onid.orst.edu> > Date: July 22, 2011 6:21:17 PM CDT > To: Mark D'Souza <dsouza(a)mcs.anl.gov> > Subject: Re: [mg-rast] Data selection errors > > Hi Mark, I was wondering if there might be a way for us to download the NCBI annotation data from the larger genomes. I've been running into the problem that you described below. The problem is that most of our metagenome samples are quite large. I know this might be a little bit more than whats expected of you, but we would really appreciate it. > > My plan of attack was this; if I provided the metagenome information (MG-RAST number), and domains of interest (all domains), I was hoping that maybe we could use drop-box, or possibly secure copy using unix to download this infomration if you would be willing to retrieve it. > > Thanks for working with us on this. > > -Anthony Bertagnolli > > On Jun 24, 2011, at 12:19 PM, Mark D'Souza wrote: > >> Hi, >> >> Thanks, that makes things clear. >> >> To get the fasta file for all organisms, you need to: >>> 1) select a metagenome from the list of options (4440275.3) >>> >>> 2) for annnotation, select Genbank (don't select any cut-offs) >>> >>> 3) hit generate table >>> >> >> change the group table to domain, >> select Bacteria (the largest set) >> >>> >>> 6) hit to work bench, moving n proteins to the workbench >>> >>> 7) move over to the work bench viewer, here, you should see two options; >>> >>> download metagenome dna FASTA annotated by : genbank >> >> Repeat for other domains, combining the smaller ones. >> >> >> In principle you should be able to select all domains and create the file for the entire dataset, however, due to a known problem, if the number of sequences is too large you will get a server error so you need to restrict to smaller sets for large metagenomes. >> >> Regards, >> Mark >> >> >> >> On Jun 24, 2011, at 11:52 AM, bertagna(a)onid.orst.edu wrote: >> >>> sure, in the metagenome analysis portion, for organism classification; >>> >>> 1) select a metagenome from the list of options (4440275.3) >>> >>> 2) for annnotation, select Genbank (don't select any cut-offs) >>> >>> 3) hit generate table >>> >>> 4) change the group table from class to species >>> >>> 5) for the species, select candidatus pelagibacter ubique (the most abundant organism in this sample) >>> >>> 6) hit to work bench, moving 1846 proteins to the workbench >>> >>> 7) move over to the work bench viewer, here, you should see two options; >>> >>> download metagenome dna FASTA annotated by : genbank >>> >>> display annotated hits below; >>> >>> both of these options allow you to view the genbank annotations for the sequences affiliated with this organism. My question is this, is there a way to download this file for ALL of the organisms detected in given sample. >>> >>> Thanks, >>> >>> -Anthony >>> >>> Quoting Mark D'Souza <dsouza(a)mcs.anl.gov>: >>> >>>> Hi, >>>> >>>> I am not sure which report you are referring to, can you tell us the steps you use to get the NCBI-nr functional report? >>>> >>>> Thanks, >>>> Mark >>>> >>>> >>>> >>>> On Jun 23, 2011, at 5:37 PM, anthony bertagnolli wrote: >>>> >>>>> Hi Mark, thanks for the reply. I think i figured out what the problem was. However, I have a second question. I noticed that there is now an option to view the NCBI-nr functional report for selected taxa. This is very useful, and i've used it quite a bit. However, I was wondering if there would be a way to download this information for whole samples, rather than just select reference taxa from samples. This would be very useful to have as well, as dowloadiing the report for ever taxa in a given sample would be too time consuming. Thanks again for all the updates to the site, its really quite a nice tool. >>>>> >>>>> -Anthony Bertagnolli >>>>> >>>>> >>>>> On Jun 15, 2011, at 4:35 PM, Mark D'Souza wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Can you tell us the mg-rast IDs for the datasets you are viewing, we will check what is the problem. >>>>>> >>>>>> Regards, >>>>>> Mark D'Souza >>>>>> -- for the MG-RAST team >>>>>> >>>>>> mg-rast(a)mcs.anl.gov >>>>>> http://metagenomics.anl.gov/ >>>>>> >>>>>> If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Jun 14, 2011, at 7:40 PM, anthony bertagnolli wrote: >>>>>> >>>>>>> Greetings, I've been using MG-RAST to analyze metagenome samples collected from marine environments, and I really like the latest version. However, I've found that when using the Data selection component to change the e-value cutt-off, % ID cuttoff, and min. alignment length, no changes are made to the corresponding data tables below. You would think that when moving from an e-value cutoff 1x10-5, from 0, or 1x10-5 to 1x10-10, you see a difference in the abundance of genes being classified (more stringent the cutt-off, the less reads assigned to a reference organism). However, this does not seem to the case. In fact, it appears as if nothing is occuring at all. Just wondering if maybe this might be a bug, and if it could be fixed. Thanks, >>>>>>> >>>>>>> -Anthony bertagnolli >>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >>> >>> >>> Anthony Bertagnolli >>> Oregon State University >>> Department of Microbiology >>> http://www.mcb.oregonstate.edu/giovannoni/projects/subproject/OMZ.htm >>> >> >> >

1 0

notification of completed jobs
by Alyson Santoro 25 Jul '11

25 Jul '11

Hello- I am new to MG-RAST . . .I uploaded some 454 data on Friday and submitted the files for annotation (I think . . .). Is there somewhere on the site that lists which jobs of mine are in the queue? Do I receive an email when the job is complete, or do I need to keep checking the website? Also, I accidentally chose a .qual file to upload and it seems to be 'stuck'. Thanks so much. Also, just a bit of feedback. When I go to the MG-RAST information pages (such as the FAQ), I lose access to the navigation icons in the upper right. Not sure if this is a browser issue. I am using Firefox 5. Thanks again, Alyson

1 0

Re: [mg-rast] MG-RAST and modelSEED
by Mark D'Souza 25 Jul '11

25 Jul '11

Hi, No, MG-RAST does not connect to the modelSEED framework, the connection is through the SEED protein and SEED subsystems databases. Regards, Mark D'Souza -- for the MG-RAST team mg-rast(a)mcs.anl.gov http://metagenomics.anl.gov/ If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov On Jul 22, 2011, at 7:30 PM, Sharon Greenblum wrote: > Hello, > I was wondering if there is an accessible connection between MG-RAST metagenomic data and the modelSEED framework? > > Thank you, > Sharon Greenblum > Graduate Student, Dept of Genome Sciences, University of Washington

1 0

Re: [mg-rast] Meaning of Abundance
by Mark DSouza 25 Jul '11

25 Jul '11

Hi, The abundance is simply the counts for the hits in the metagenome sequences against the selected databases. The website displays unique hits, while the downloaded fasta file contains non-unique entries with one entry for each database annotation, so the numbers will generally not match. The read ID is embedded in the sequence ID used in the fasta file, if you extract and count them the numbers should match. In your case the numbers do seem excessive, which could be the case if you are using the M5NR database which encompasses multiple databases. If you can not reconcile the numbers let us know, and include the MG-RAST ID for the dataset with a description of the steps you took. -- Regards, Mark D'Souza -- for the MG-RAST team mg-rast(a)mcs.anl.gov http://metagenomics.anl.gov/ If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov ----- Original Message ----- > From: "Matthew T. Agler" <mta42(a)cornell.edu> > To: mg-rast(a)mcs.anl.gov > Sent: Monday, July 25, 2011 10:28:48 AM > Subject: [mg-rast] Meaning of Abundance > Dear MG-RAST team; > > We are curious of the meaning of abundance in, for example, a > functional annotation table. When I export a single gene to the > workbench then download the corresponding fasta file, there is > typically far more sequences in the fasta file than were given for > abundance. For example, I just downloaded a gene with abundance ~4000 > and there were ~60,000 sequences in the corresponding fasta file, an > order of magnitude difference. > > Is there some sort of clustering going on? If so, how can we compare > relative abundances for samples if clustering may have been different? > > Thanks! > > Matt Agler > > -- > Matthew T. Agler, Graduate Student Assistant > Cornell University > Riley-Robb Hall > Ithaca, NY 14853 > > ph: 314-625-2633

1 0

Meaning of Abundance
by Matthew T. Agler 25 Jul '11

25 Jul '11

Dear MG-RAST team; We are curious of the meaning of abundance in, for example, a functional annotation table. When I export a single gene to the workbench then download the corresponding fasta file, there is typically far more sequences in the fasta file than were given for abundance. For example, I just downloaded a gene with abundance ~4000 and there were ~60,000 sequences in the corresponding fasta file, an order of magnitude difference. Is there some sort of clustering going on? If so, how can we compare relative abundances for samples if clustering may have been different? Thanks! Matt Agler -- Matthew T. Agler, Graduate Student Assistant Cornell University Riley-Robb Hall Ithaca, NY 14853 ph: 314-625-2633

1 0

MG-RAST and modelSEED
by Sharon Greenblum 22 Jul '11

22 Jul '11

Hello, I was wondering if there is an accessible connection between MG-RAST metagenomic data and the modelSEED framework? Thank you, Sharon Greenblum *Graduate Student, Dept of Genome Sciences, University of Washington*

1 0

Re: [mg-rast] Sharing MG-RAST jobs
by Mark DSouza 22 Jul '11

22 Jul '11

Hi Steve, The jobs are now shared, sorry for the delay. Mark ----- Original Message ----- > From: "Steve Essinger" <sde22(a)drexel.edu> > To: "Mark DSouza" <dsouza(a)mcs.anl.gov> > Sent: Wednesday, July 20, 2011 2:04:58 PM > Subject: Re: Sharing MG-RAST jobs > Thanks so much, Mark. > > > On Wed, Jul 20, 2011 at 3:02 PM, Mark DSouza < dsouza(a)mcs.anl.gov > > wrote: > > > Hi Steve, > > Sorry, I have not been able to get this done yet, I plan to get it > done by tomorrow. > > Mark > > > > > ----- Original Message ----- > > From: "Steve Essinger" < sde22(a)drexel.edu > > > To: "Mark D'Souza" < dsouza(a)mcs.anl.gov > > > Sent: Wednesday, July 20, 2011 1:06:11 PM > > Subject: Re: Sharing MG-RAST jobs > > Hi Mark, > > > > > > Were you able to share the 90 datasets with these three individuals? > > > > > > Thanks, > > Steve > > > > > > On Mon, Jul 18, 2011 at 6:42 PM, Steve Essinger < sde22(a)drexel.edu > > > wrote: > > > > > > > > > > > > > > > > > > On Mon, Jul 18, 2011 at 6:31 PM, Mark D'Souza < dsouza(a)mcs.anl.gov > > > wrote: > > > > > > Hi, > > > > Please reply to this email, no additional text necessary. > > > > Share all 90 datasets with: > > > empr3ss(a)gmail.com > > > uh25(a)drexel.edu > > > gcd34(a)drexel.edu > > > > Regards, > > Mark > > > > > > > > ----- Original Message ----- > > > From: "Steve Essinger" < sessinger(a)gmail.com > > > > To: mg-rast(a)mcs.anl.gov > > > Sent: Monday, July 18, 2011 3:54:12 PM > > > Subject: [mg-rast] Sharing Datasets with Multiple Users > > > Hi, > > > > > > > > > I currently have 90 datasets uploaded on MG-RAST. I need to share > > > these datasets with a few individuals, but cannot give them my > > > username and password. I see that here is a link to share > > > datasets, > > > but each one must be shared individual with each person. This is > > > going > > > to take a long time. Is there anyway to share my datasets as a > > > batch > > > process? If not, is it an easy fix for you to share the datasets > > > for > > > me? > > > > > > > > > My username is: sessinger > > > The datasets need to be shared with: > > > empr3ss(a)gmail.com > > > uh25(a)drexel.edu > > > gcd34(a)drexel.edu > > > > > > > > > Thanks, > > > Steve Essinger

1 0

Re: [mg-rast] bug for tree generation
by Mark DSouza 22 Jul '11

22 Jul '11

Hi, Unfortunately you have come up against a limit on the number of proteins that can be used to create the tree, at some point the number gets too large and the time taken for data retrieval causes timeouts. Can you reduce the number of proteins used to create the tree for your analysis? Possibly create multiple trees at the phylum or class level. Regards, Mark ----- Original Message ----- > From: amarots(a)iqsc.usp.br > To: "Mark D'Souza" <dsouza(a)mcs.anl.gov> > Sent: Friday, July 22, 2011 11:28:35 AM > Subject: Re: [mg-rast] bug for tree generation > Hi Mark, > I wonder if you gave a look at the bug I mentioned before? > I've been trying to build this tree fig. but still can't get it to > work. > Thanks, > Amaro. > > > > > > Hi, > > > > The database query required to create the trees becomes more complex > > with > > number of metagenomes and increasing number of filtering steps > > causing > > lengthy times for retrieval of data and may cause the tree > > visualization > > to fail, we will check and see what can be done. > > > > Regards, > > Mark D'Souza > > -- for the MG-RAST team > > > > mg-rast(a)mcs.anl.gov > > http://metagenomics.anl.gov/ > > > > If you need to respond to this email please address it to the > > mailing-list > > mg-rast(a)mcs.anl.gov > > > > > > > > > > On Jul 19, 2011, at 9:03 AM, amarots(a)iqsc.usp.br wrote: > > > >> Hello MG-RAST team, > >> I wonder if you could give me a help. I can't visualize a specific > >> data > >> set saved at workbench as a tree scheme. I picked metagenomes: > >> 4461456.3, > >> 4461455.3, 4461454.3, 4461453.3,4461452.3, 4461451.3, 4453374.3, > >> 4461444.3, and pooled the Bacteria domain related proteins (total > >> of > >> 48557 > >> unique proteins, using GenBank and cutoff of 1e-5 and Min Alig > >> Length of > >> 50bp). When I try to generate a tree using this set of ptns the > >> "data > >> generation progress" keep going forever. When I use all ptns info > >> from > >> this group of metagemones, a shorter workbench set of ptns or less > >> metagenomes I could get the tree scheme tough. So, it seems to be a > >> bug > >> that happens specificaly with the set of ptns that I chose. > >> Thanks, > >> Amaro > >> > >> > >> > >> > > > >

1 0

error in jobs:
by Laura Hug 22 Jul '11

22 Jul '11

Hello MG-RAST folk! I uploaded two projects for annotation earlier this week, and it appears there is an error at the sequence screening step. This is for jobs 28190 and 28019. I can try to fix the input files, but am not sure what the error is. Rather than just try again, I figured I would touch base with you to see if there is an easy fix from my side. It also appears that job 27188 has stalled: it has been at the "loading database: complete" stage for approximately two weeks now. Is there anything wrong with this job? Thanks! cheers, Laura

1 0

Re: [mg-rast] groups on MG-RAST
by Mark DSouza 21 Jul '11

21 Jul '11

Hi, We are reworking the whole sharing mechanism but right now it is rather cumbersome. You are right, the groups can be used to share metagenomes, let me know the name of the group and I can create it for you and make you the administrator, you can then add members and metagenomes. -- Regards, Mark D'Souza -- for the MG-RAST team mg-rast(a)mcs.anl.gov http://metagenomics.anl.gov/ If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov ----- Original Message ----- > From: "Tracy Teal" <tracyt(a)idyll.org> > To: mg-rast(a)mcs.anl.gov > Sent: Thursday, July 21, 2011 10:28:39 AM > Subject: [mg-rast] groups on MG-RAST > Hi, I'm sure I'm missing something obvious in the documentation, but I > was wondering how to create membership groups and if we can use those > groups to share metagenomes? We have several metagenomes that we want > to share with multiple people in the lab, and it would be easiest to > provide group access and then add memebers to that group, rather than > having to individually add a new person to each metagenome. > > Thanks so much for your help. > > -Tracy > > --- > Tracy K. Teal > Postdoctoral researcher > T. Schmidt laboratory > Michigan State University > tkteal(a)msu.edu

1 0