July 2011 - Mg-rast - mailman3.cels.anl.gov

Possible bug with KEGG Mapper
by sophie.davidson＠dpi.vic.gov.au 13 Jul '11

13 Jul '11

Hi, I've been trying to use the KEGG mapper option to visualise my data. I've been finding some inconsistancies. When I upload only one set of metagenomes into either 'Data A' or 'Data B' with the other Data A/B left empty and highlight the loaded data it is totally different to what I get highlighted when I have two data sets loaded (i.e. something in Data A and something in Data B) and I ask to have only Data A or Data B highlighted. This happens even when the parameters are exactly the same Regards, Sophie Sophie Davidson | BioScience Researcher | BioScience Division Department of Primary Industries | Victorian AgriBiosciences Centre | 1 Park Dr | Bundoora VIC 3083 T: 03 9032 7111 | F: 03 9479 3618 | E: sophie.davidson(a)dpi.vic.gov.au Notice: This email and any attachments may contain information that is personal, confidential, legally privileged and/or copyright.No part of it should be reproduced, adapted or communicated without the prior written consent of the copyright owner. It is the responsibility of the recipient to check for and remove viruses. If you have received this email in error, please notify the sender by return email, delete it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information contained in this email. Please consider the environment before printing this email.

1 0

Re: [mg-rast] re job hanging
by Mark DSouza 13 Jul '11

13 Jul '11

Hi, There was a problem with the demultiplexing, I ran this step and all your sequences got classified under the 'no_MID_tag' status as all the sequences in the sff file begin with 'tcag' which does not match any of the MID tags. You can modify your MID tags to include this prefix and resubmit the dataset, let me know if you have trouble with the demultiplexing again. Regards, Mark ----- Original Message ----- > From: "Julian Marchesi" <MarchesiJR(a)cardiff.ac.uk> > To: "Mark D'Souza" <dsouza(a)mcs.anl.gov> > Sent: Tuesday, July 12, 2011 9:41:34 AM > Subject: Re: [mg-rast] re job hanging > cheers > > Julian > > > > Dr Julian R. Marchesi > School of Biosciences > Museum Avenue > Cardiff University > Cardiff > CF10 3AT > Telephone: +44 (0)29 208 74188 > Fax: +44 (0)29 20874305 > Mobile 07885 569144 > > The information contained in this e-mail message and any attachments > is strictly confidential and is intended for the person or > organisation to whom it is addressed. > If you are not the intended recipient, you must not read, use, copy or > disseminate the information contained in this message and must delete > it. Please advise us > immediately if you are not the named addresses or the person > responsible for delivering the message to the named addressee. > > Although Cardiff University believes this e-mail and any attachments > are free of any virus or defect which may affect a computer, it is the > responsibility of the recipient > to ensure that it is virus free. Cardiff university will not accept > any liability for any loss or damage arising in any way from its use. > > > > From: Mark D'Souza <dsouza(a)mcs.anl.gov> > To: Julian Marchesi <MarchesiJR(a)cardiff.ac.uk> > Cc: mg-rast(a)mcs.anl.gov > Date: 12/07/2011 15:30 > Subject: Re: [mg-rast] re job hanging > > > > > Hi, > > No, that is not normal, we will take a look and get back to you. > > Regards, > Mark D'Souza > -- for the MG-RAST team > > mg-rast(a)mcs.anl.gov > http://metagenomics.anl.gov/ > > If you need to respond to this email please address it to the > mailing-list mg-rast(a)mcs.anl.gov > > > > > On Jul 12, 2011, at 8:18 AM, Julian Marchesi wrote: > > > Hi Guys > > > > I've uploaded my data (16S rRNA gene) and have supplied the MIDS and > > have been waiting >24 hours for it to demultiplex > > > > job number 42129 here is the status > > > > my login name is jugsy67 > > > > filename status next step progress > > GPFRVDY05.sff.fasta demultiplex demultiplexing waiting for > > demultiplexing ... > > > > Is this normal? The file is an sff file approx 216 Mb in size > > > > Dr Julian R. Marchesi > > School of Biosciences > > Museum Avenue > > Cardiff University > > Cardiff > > CF10 3AT > > Telephone: +44 (0)29 208 74188 > > Fax: +44 (0)29 20874305 > > Mobile 07885 569144 > > > > The information contained in this e-mail message and any attachments > > is strictly confidential and is intended for the person or > > organisation to whom it is addressed. > > If you are not the intended recipient, you must not read, use, copy > > or disseminate the information contained in this message and must > > delete it. Please advise us > > immediately if you are not the named addresses or the person > > responsible for delivering the message to the named addressee. > > > > Although Cardiff University believes this e-mail and any attachments > > are free of any virus or defect which may affect a computer, it is > > the responsibility of the recipient > > to ensure that it is virus free. Cardiff university will not accept > > any liability for any loss or damage arising in any way from its > > use.

1 0

Re: [mg-rast] HI... wondering if either I or your program screwed up
by Mark D'Souza 13 Jul '11

13 Jul '11

Hi, The computation stages for all the jobs submitted to MG-RAST get queued on first-come first-served basis and your jobs are queued behind other jobs which is why they are taking so long, the actual computation time is negligible. The system does not do well with extremely small jobs and the small job (27349) will not complete, the other one (27342) is progressing, but slowly. You may want to try a test of several Mbp of sequence data, even upto 50 Mbp, to try out the statistical tools in MG-RAST. Regards, Mark On Jul 12, 2011, at 5:10 PM, Peter Noble wrote: > Hello Mark, > > Sorry to bother you again. I suspect that there is something wrong with either my data or your program. I followed the directions exactly as written. I put in a small job to prove to myself that there is in fact something wrong. Both the medium and small job are stalled. > > Could you please take a look at my files to tell me what is going wrong. Shouldn't a small file be processed in a couple of hours? > > The purpose of submitting my files is just to see if your approach can be used for all my 454 sequences. > > Thanks for your time. > > Peter > On Jul 11, 2011, at 2:26 PM, Mark D'Souza wrote: > >> Hi, >> >> The files I see for these jobs are very small, job 27342, 4467745.3, test is 4.4M and contains 906 sequences and job 27349 is smaller. I don't see anything wrong with the upload, could you check that you uploaded the correct files? >> >> As the FAQ states, the numbers are for optimal conditions, namely when there are no queued jobs. Right now we are recovering from a power outage which occurred last week and required us to restart a number of jobs leading to some delay. >> >> Regards, >> Mark D'Souza >> -- for the MG-RAST team >> >> mg-rast(a)mcs.anl.gov >> http://metagenomics.anl.gov/ >> >> If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov >> >> >> >> >> On Jul 11, 2011, at 1:59 PM, Peter Noble wrote: >> >>> Hello, >>> >>> I am very excited about analyzing my 454 sequence data with your software. I submitted two test files to see how your program works. One file contained 4.4 million reads and the other one was just 0.3 million reads. I have gotten no response after several hours. According to your FAQ it would take about 18 hours to run a 100 million bp through the pipeline. >>> >>> Have I done something wrong? The little green indicator buttons say my analyses are only half done and "Dereplication : completed" >>> >>> Please advise. >>> >>> Peter >>> >>> >>> >>> >>> >>> >>> Peter A. Noble >>> Associate Professor of Microbiology >>> Alabama State University >>> Montgomery Alabama 36104 >>> >>> Affiliate Associate Professor of Periodontics >>> University of Washington >>> Seattle Washington 98195 >>> >>> Email: panoble(a)washington.edu >>> Phone: 334-229-8451 >>> http://peteranoble.com >>> >>> >>> >>> >> >

1 0

Re: [mg-rast] about mg-rast uploading
by Mark D'Souza 13 Jul '11

13 Jul '11

Hi, Yes, the dataset is suitable as far as the size. The MG-RAST analysis requires read lengths of 75 bp or more, so again at 76 bp your dataset will be fine. Regards, Mark On Jul 12, 2011, at 4:33 AM, Jie Feng wrote: > I have a compressed fastq dataset up to 4.3Gb in size(11.fastq.gz) and it is sequnenced by Illumina sequencers. About five days ago, I uploaded it to mg-rast. But the processing step has never started. And you said the upload process was not completed correctly. What I want to know is whether my dataset is suitable for your server. > Thank youvery much! > > > > > > >

1 0

Re: [mg-rast] Errors on processing
by Mark DSouza 13 Jul '11

13 Jul '11

Hi, The jobs are queued, including the ones showing the error status, they are not showing progress because the queue is moving rather slowly right now as we work through a backlog. Regards, Mark ----- Original Message ----- > From: "Matthew T. Agler" <mta42(a)cornell.edu> > To: "Mark D'Souza" <dsouza(a)mcs.anl.gov> > Sent: Wednesday, July 13, 2011 7:44:36 AM > Subject: Errors on processing > Hi Mark; > > Most of my samples have made it through the pipeline. Four of them > (27127, 27128, 27122, and 27094), however, don't seem to be moving. > Particularly, the two I mentioned last week (jobs 27127 and 27128) > that say they have errors on them are still in the same place with the > same error message. Would it be best just to upload them again? > > Thanks > Matt > > -- > Matthew T. Agler > Cornell University > Riley-Robb Hall > Ithaca, NY 14853 > > ph: 314-625-2633

1 0

Re: [mg-rast] stuff
by Jack Gilbert 13 Jul '11

13 Jul '11

ok - see you at 2:30 J On Wed, Jul 13, 2011 at 10:46 AM, Folker Meyer <folker(a)anl.gov> wrote: > Hi Jack: > > why don't you come over this afternoon and we talk about it? > > I am meeting with Gene and another person at 1pm so make it 2:30 and come > over? > > meeting in my office? diet coke is on me > > Cheers, > Folker > > > > On Jul 12, 2011, at 4:41 PM, Jack Gilbert wrote: > > Hey guys, > > a few of things: > > (1) we need to figure out the appropriate submission tools for dealing with > the two fastaq output from Illumina machines. Greg has dealt with this > before. Do you want me to talk to him about this? I am not sure what they > have in place but it is not available on the public QIIME version, they are > currently doing it all behind the scenes. > > (2) The reason there are two separate fastq files is because the sequencing > machine generates two separate reads for amplicon sequencing in this format > - there is nothing we can do about that - it is not cutting the barcode off > and sending it in a separate file, it is generate two sequencing runs per > sequence. > > (3) Projects - can we set aside a 30 minutes tomorrow for me to come over > with Sarah and tell you which existing data in mgrast need to go into which > projects, and which data about to be uploaded into mgrast need to go into > which projects. > > I think all of this will be very helpful. > > Cheers > > jack > > -- > Dr. Jack A. Gilbert B.Sc. Ph.D. > Argonne National Laboratory > Bldg. 202, Biosciences > Room B373 > 9700 South Cass Avenue > Argonne, IL, 60430 > Tel: +1 630 252-7489 > Cell: +1 630 915 2383 > Email: gilbertjack(a)anl.gov <gilbertjack(a)gmail.com> > > > -- Dr. Jack A. Gilbert B.Sc. Ph.D. Argonne National Laboratory Bldg. 202, Biosciences Room B373 9700 South Cass Avenue Argonne, IL, 60430 Tel: +1 630 252-7489 Cell: +1 630 915 2383 Email: gilbertjack(a)anl.gov <gilbertjack(a)gmail.com>

1 0

Re: [mg-rast] stuff
by Folker Meyer 13 Jul '11

13 Jul '11

Hi Jack: why don't you come over this afternoon and we talk about it? I am meeting with Gene and another person at 1pm so make it 2:30 and come over? meeting in my office? diet coke is on me Cheers, Folker On Jul 12, 2011, at 4:41 PM, Jack Gilbert wrote: > Hey guys, > > a few of things: > > (1) we need to figure out the appropriate submission tools for dealing with the two fastaq output from Illumina machines. Greg has dealt with this before. Do you want me to talk to him about this? I am not sure what they have in place but it is not available on the public QIIME version, they are currently doing it all behind the scenes. > > (2) The reason there are two separate fastq files is because the sequencing machine generates two separate reads for amplicon sequencing in this format - there is nothing we can do about that - it is not cutting the barcode off and sending it in a separate file, it is generate two sequencing runs per sequence. > > (3) Projects - can we set aside a 30 minutes tomorrow for me to come over with Sarah and tell you which existing data in mgrast need to go into which projects, and which data about to be uploaded into mgrast need to go into which projects. > > I think all of this will be very helpful. > > Cheers > > jack > > -- > Dr. Jack A. Gilbert B.Sc. Ph.D. > Argonne National Laboratory > Bldg. 202, Biosciences > Room B373 > 9700 South Cass Avenue > Argonne, IL, 60430 > Tel: +1 630 252-7489 > Cell: +1 630 915 2383 > Email: gilbertjack(a)anl.gov

1 0

Re: [mg-rast] Progress stuck at "Dereplication : completed"?
by Mark D'Souza 13 Jul '11

13 Jul '11

Hi, This is perfectly normal, plus we have a backlog in the compute queue due to a power outage last week, it would usually take a couple of days for a job to get completed. Regards, Mark D'Souza -- for the MG-RAST team mg-rast(a)mcs.anl.gov http://metagenomics.anl.gov/ If you need to respond to this email please address it to the mailing-list mg-rast(a)mcs.anl.gov On Jul 13, 2011, at 9:35 AM, Stephen Mondo wrote: > Dear mg-rast, > My name is Stephen Mondo (login: smondo1). I have recently > uploaded a dataset which is not yet ready for analysis. I uploaded > this file last night (approx. 14 hours ago), but the progress seems to > be halted at "Dereplication : completed". Is this normal? How long > should I expect to wait until my data are ready for analysis? > > Thank you very much for your help!, > Stephen

1 0

Progress stuck at "Dereplication : completed"?
by Stephen Mondo 13 Jul '11

13 Jul '11

Dear mg-rast, My name is Stephen Mondo (login: smondo1). I have recently uploaded a dataset which is not yet ready for analysis. I uploaded this file last night (approx. 14 hours ago), but the progress seems to be halted at "Dereplication : completed". Is this normal? How long should I expect to wait until my data are ready for analysis? Thank you very much for your help!, Stephen

1 0

stuff
by Jack Gilbert 12 Jul '11

12 Jul '11

Hey guys, a few of things: (1) we need to figure out the appropriate submission tools for dealing with the two fastaq output from Illumina machines. Greg has dealt with this before. Do you want me to talk to him about this? I am not sure what they have in place but it is not available on the public QIIME version, they are currently doing it all behind the scenes. (2) The reason there are two separate fastq files is because the sequencing machine generates two separate reads for amplicon sequencing in this format - there is nothing we can do about that - it is not cutting the barcode off and sending it in a separate file, it is generate two sequencing runs per sequence. (3) Projects - can we set aside a 30 minutes tomorrow for me to come over with Sarah and tell you which existing data in mgrast need to go into which projects, and which data about to be uploaded into mgrast need to go into which projects. I think all of this will be very helpful. Cheers jack -- Dr. Jack A. Gilbert B.Sc. Ph.D. Argonne National Laboratory Bldg. 202, Biosciences Room B373 9700 South Cass Avenue Argonne, IL, 60430 Tel: +1 630 252-7489 Cell: +1 630 915 2383 Email: gilbertjack(a)anl.gov <gilbertjack(a)gmail.com>

1 0