August 2011 - Mg-rast - mailman3.cels.anl.gov

Re: [mg-rast] problem with very large sff file uploaded and processed incorrectly
by William Trimble 01 Aug '11

01 Aug '11

Oh, that's probably the reason, then. Concatenating SFF files requires programs that deliberately write SFF files. In that case, you could either send us the two SFF files separately and run them as two separate jobs or convert both runs to FASTQ and upload the concatenated FASTQ. Either one will work with MG-RAST. William Trimble --for the MG-RAST team On Mon, Aug 1, 2011 at 1:40 PM, Alison Murray <Alison.Murray(a)dri.edu> wrote: > OK, > > will start that shortly. I did combine two sff files into the one that I > uploaded; I wonder if that caused the problem? I wanted to join all the > sequence I have from this data set into one analysis - we had a small set of > data from the titration runs, and then the big main data set. The file that > I have to upload is the combined one (I concatenated the two sff files). > Actually - if you think that's a problem, let me know and I'll do them > separately. It's all the same library though so would like the analysis > ideally to consider all the sequence files ran in these two batches from the > same data set. > > -Alison > > > > ----- Original Message ----- > From: William Trimble <wltrimbl(a)gmail.com> > Date: Monday, August 1, 2011 11:35 am > Subject: Re: [mg-rast] problem with very large sff file uploaded and > processed incorrectly > To: Alison Murray <Alison.Murray(a)dri.edu> > >> It appears that we don't have an intact copy >> of your dataset. >> >> Yes, you should just upload the file, let us >> know, and if it looks ok you can create a new job. >> >> William Trimble >> --for the MG-RAST team >> >> >> On Mon, Aug 1, 2011 at 1:25 PM, Alison Murray >> <Alison.Murray(a)dri.edu> wrote: >> > Hi William, >> > >> > The numbers are slightly different from what I see. I think >> that I >> > compressed a 7,539,547 SFF file - such that the one that was >> uploaded was >> > 3,651,001. (File name Norsel3MG_all.sff). >> > >> > Should I go ahead and try and upload through the MGRAST >> browser this >> > compressed file? >> > >> > From what you suggest - I think the answer is yes, but am not >> sure of the >> > differences in file size. >> > >> > -Alison >> > >> > >> > >> > >> > ----- Original Message ----- >> > From: William Trimble <wltrimbl(a)gmail.com> >> > Date: Monday, August 1, 2011 11:18 am >> > Subject: Re: [mg-rast] problem with very large sff file >> uploaded and >> > processed incorrectly >> > To: Alison Murray <Alison.Murray(a)dri.edu>, mg-rast(a)mcs.anl.gov >> > >> >> Thank you for letting us know you're having trouble. >> >> >> >> You uploaded a compressed 7 Gb SFF file, but >> >> the file had a problem that caused us to find >> >> only 74 reads in it. >> >> >> >> This problem was probably caused by an error during the >> >> upload process. >> >> >> >> The SFF file that we got was 7720495172 bytes long. >> >> does this match the size of the file that you sent? >> >> >> >> You will need to upload the compressed SFF file again. >> >> We can then check to make sure it is complete. >> >> >> >> William Trimble >> >> --for the MG-RAST team >> >> >> >> >> >> On Mon, Aug 1, 2011 at 12:40 PM, Alison Murray >> >> <Alison.Murray(a)dri.edu> wrote: >> >> > Hi there, >> >> > >> >> > I uploaded a very large file - over 3.6GB that was a sff >> file (name: >> >> > Norsel3-S_adreanum_MicrobiomeMG) a few weeks ago - this was >> >> submitted for >> >> > analysis - and today I checked, and it was "available for >> analysis".>> > However, it appears that only a very small >> portion of the >> >> sequences were >> >> > actually uploaded/processed. I'm wondering if somehow the >> >> criteria for QC >> >> > was too strict? Or if I crashed the server??? Really have no >> >> clue as to why >> >> > it didn't work - but definitely a problem here. This is a >> >> huge metagenome >> >> > data set - generated with 454. >> >> > >> >> > Would appreciate it if someone could help me with this - as >> >> it's literally >> >> > be 3 weeks waiting to then see that something didn't work >> >> right - in the >> >> > upload/processing settings were mostly the default settings as >> >> far as I >> >> > recall. I uploaded the raw, non-assembled data set. (supposed >> >> to be 865 MB >> >> > of raw sequence) >> >> > >> >> > Thanks for the help! >> >> > >> >> > Alison >> >> > >> >> > >> >> > >> >> > Alison E. Murray, Ph.D.// >> >> > Associate Research Professor// >> >> > Desert Research Institute// >> >> > 2215 Raggio Parkway// >> >> > Reno, NV 89512// >> >> > >> >> > Tel: 775 673 7361// >> >> > FAX: 775 673 7635// >> >> > http://www.dri.edu/People/alison/ >> >> > >> >> > >> > >> > Alison E. Murray, Ph.D.// >> > Associate Research Professor// >> > Desert Research Institute// >> > 2215 Raggio Parkway// >> > Reno, NV 89512// >> > >> > Tel: 775 673 7361// >> > FAX: 775 673 7635// >> > http://www.dri.edu/People/alison/ >> > >> > > > Alison E. Murray, Ph.D.// > Associate Research Professor// > Desert Research Institute// > 2215 Raggio Parkway// > Reno, NV 89512// > > Tel: 775 673 7361// > FAX: 775 673 7635// > http://www.dri.edu/People/alison/ > >

1 0

Fwd: problem with very large sff file uploaded and processed incorrectly
by William Trimble 01 Aug '11

01 Aug '11

It appears that we don't have an intact copy of your dataset. Yes, you should just upload the file, let us know, and if it looks ok you can create a new job. William Trimble --for the MG-RAST team On Mon, Aug 1, 2011 at 1:25 PM, Alison Murray <Alison.Murray(a)dri.edu> wrote: > Hi William, > > The numbers are slightly different from what I see. I think that I > compressed a 7,539,547 SFF file - such that the one that was uploaded was > 3,651,001. (File name Norsel3MG_all.sff). > > Should I go ahead and try and upload through the MGRAST browser this > compressed file? > > From what you suggest - I think the answer is yes, but am not sure of the > differences in file size. > > -Alison > > > > > ----- Original Message ----- > From: William Trimble <wltrimbl(a)gmail.com> > Date: Monday, August 1, 2011 11:18 am > Subject: Re: [mg-rast] problem with very large sff file uploaded and > processed incorrectly > To: Alison Murray <Alison.Murray(a)dri.edu>, mg-rast(a)mcs.anl.gov > >> Thank you for letting us know you're having trouble. >> >> You uploaded a compressed 7 Gb SFF file, but >> the file had a problem that caused us to find >> only 74 reads in it. >> >> This problem was probably caused by an error during the >> upload process. >> >> The SFF file that we got was 7720495172 bytes long. >> does this match the size of the file that you sent? >> >> You will need to upload the compressed SFF file again. >> We can then check to make sure it is complete. >> >> William Trimble >> --for the MG-RAST team >> >> >> On Mon, Aug 1, 2011 at 12:40 PM, Alison Murray >> <Alison.Murray(a)dri.edu> wrote: >> > Hi there, >> > >> > I uploaded a very large file - over 3.6GB that was a sff file (name: >> > Norsel3-S_adreanum_MicrobiomeMG) a few weeks ago - this was >> submitted for >> > analysis - and today I checked, and it was "available for analysis". >> > However, it appears that only a very small portion of the >> sequences were >> > actually uploaded/processed. I'm wondering if somehow the >> criteria for QC >> > was too strict? Or if I crashed the server??? Really have no >> clue as to why >> > it didn't work - but definitely a problem here. This is a >> huge metagenome >> > data set - generated with 454. >> > >> > Would appreciate it if someone could help me with this - as >> it's literally >> > be 3 weeks waiting to then see that something didn't work >> right - in the >> > upload/processing settings were mostly the default settings as >> far as I >> > recall. I uploaded the raw, non-assembled data set. (supposed >> to be 865 MB >> > of raw sequence) >> > >> > Thanks for the help! >> > >> > Alison >> > >> > >> > >> > Alison E. Murray, Ph.D.// >> > Associate Research Professor// >> > Desert Research Institute// >> > 2215 Raggio Parkway// >> > Reno, NV 89512// >> > >> > Tel: 775 673 7361// >> > FAX: 775 673 7635// >> > http://www.dri.edu/People/alison/ >> > >> > > > Alison E. Murray, Ph.D.// > Associate Research Professor// > Desert Research Institute// > 2215 Raggio Parkway// > Reno, NV 89512// > > Tel: 775 673 7361// > FAX: 775 673 7635// > http://www.dri.edu/People/alison/ > >

1 0

Re: [mg-rast] problem with very large sff file uploaded and processed incorrectly
by William Trimble 01 Aug '11

01 Aug '11

Thank you for letting us know you're having trouble. You uploaded a compressed 7 Gb SFF file, but the file had a problem that caused us to find only 74 reads in it. This problem was probably caused by an error during the upload process. The SFF file that we got was 7720495172 bytes long. does this match the size of the file that you sent? You will need to upload the compressed SFF file again. We can then check to make sure it is complete. William Trimble --for the MG-RAST team On Mon, Aug 1, 2011 at 12:40 PM, Alison Murray <Alison.Murray(a)dri.edu> wrote: > Hi there, > > I uploaded a very large file - over 3.6GB that was a sff file (name: > Norsel3-S_adreanum_MicrobiomeMG) a few weeks ago - this was submitted for > analysis - and today I checked, and it was "available for analysis". > However, it appears that only a very small portion of the sequences were > actually uploaded/processed. I'm wondering if somehow the criteria for QC > was too strict? Or if I crashed the server??? Really have no clue as to why > it didn't work - but definitely a problem here. This is a huge metagenome > data set - generated with 454. > > Would appreciate it if someone could help me with this - as it's literally > be 3 weeks waiting to then see that something didn't work right - in the > upload/processing settings were mostly the default settings as far as I > recall. I uploaded the raw, non-assembled data set. (supposed to be 865 MB > of raw sequence) > > Thanks for the help! > > Alison > > > > Alison E. Murray, Ph.D.// > Associate Research Professor// > Desert Research Institute// > 2215 Raggio Parkway// > Reno, NV 89512// > > Tel: 775 673 7361// > FAX: 775 673 7635// > http://www.dri.edu/People/alison/ > >

1 0

problem with very large sff file uploaded and processed incorrectly
by Alison Murray 01 Aug '11

01 Aug '11

Hi there, I uploaded a very large file - over 3.6GB that was a sff file (name: Norsel3-S_adreanum_MicrobiomeMG) a few weeks ago - this was submitted for analysis - and today I checked, and it was "available for analysis". However, it appears that only a very small portion of the sequences were actually uploaded/processed. I'm wondering if somehow the criteria for QC was too strict? Or if I crashed the server??? Really have no clue as to why it didn't work - but definitely a problem here. This is a huge metagenome data set - generated with 454. Would appreciate it if someone could help me with this - as it's literally be 3 weeks waiting to then see that something didn't work right - in the upload/processing settings were mostly the default settings as far as I recall. I uploaded the raw, non-assembled data set. (supposed to be 865 MB of raw sequence) Thanks for the help! Alison Alison E. Murray, Ph.D.// Associate Research Professor// Desert Research Institute// 2215 Raggio Parkway// Reno, NV 89512// Tel: 775 673 7361// FAX: 775 673 7635// http://www.dri.edu/People/alison/

1 0

Fwd: job not progressing
by William Trimble 01 Aug '11

01 Aug '11

---------- Forwarded message ---------- From: Layton, Alice Camille <alayton(a)utk.edu> Date: Mon, Aug 1, 2011 at 11:07 AM Subject: RE: [mg-rast] job not progressing To: William Trimble <wltrimbl(a)gmail.com> Thanks for the help. Next time I will add the CTAG to the barcode. -----Original Message----- From: William Trimble [mailto:[email protected]] Sent: Monday, August 01, 2011 12:06 PM To: Layton, Alice Camille; mg-rast(a)mcs.anl.gov Subject: Re: [mg-rast] job not progressing The problem in demultiplexing was that your sequences begin with CTAG + Barcode and the demultiplexer was looking for sequences that began with Barcode The samples can be run through by 1) prepending CTAG to each barcode or by 2) removing the CTAG prefix from each sequence We will have to remove the stuck job for you. Thanks for letting us know about the problem. William Trimble --for the MG-RAST team On Mon, Aug 1, 2011 at 9:23 AM, Layton, Alice Camille <alayton(a)utk.edu> wrote: > Thanks, If it doesn't work you can delete the job. I found that I can upload the sequences as fasta files after separation into individual files through the RDP pipeline. > > -----Original Message----- > From: William Trimble [mailto:[email protected]] > Sent: Monday, August 01, 2011 10:03 AM > To: Layton, Alice Camille > Subject: Re: [mg-rast] job not progressing > > We're looking into it. > > William Trimble > --for the MG-RAST team > > On Mon, Aug 1, 2011 at 7:24 AM, Layton, Alice Camille <alayton(a)utk.edu> wrote: >> Dear rast, >> >> >> >> I started this job last Thursday but it seems stuck at this step >> >> >> >> G6J8P7S01.sff_no_MID_tag.fasta demultiplex demultiplexing waiting for >> demultiplexing .... I can't delete it and I don't get any other >> windows to start the job. >> >> >> >> Should I re-upload it? >> >> >> >> Thanks, >> >> >> >> Alice Layton > > >

1 0

Re: [mg-rast] job not progressing
by William Trimble 01 Aug '11

01 Aug '11

The problem in demultiplexing was that your sequences begin with CTAG + Barcode and the demultiplexer was looking for sequences that began with Barcode The samples can be run through by 1) prepending CTAG to each barcode or by 2) removing the CTAG prefix from each sequence We will have to remove the stuck job for you. Thanks for letting us know about the problem. William Trimble --for the MG-RAST team On Mon, Aug 1, 2011 at 9:23 AM, Layton, Alice Camille <alayton(a)utk.edu> wrote: > Thanks, If it doesn't work you can delete the job. I found that I can upload the sequences as fasta files after separation into individual files through the RDP pipeline. > > -----Original Message----- > From: William Trimble [mailto:[email protected]] > Sent: Monday, August 01, 2011 10:03 AM > To: Layton, Alice Camille > Subject: Re: [mg-rast] job not progressing > > We're looking into it. > > William Trimble > --for the MG-RAST team > > On Mon, Aug 1, 2011 at 7:24 AM, Layton, Alice Camille <alayton(a)utk.edu> wrote: >> Dear rast, >> >> >> >> I started this job last Thursday but it seems stuck at this step >> >> >> >> G6J8P7S01.sff_no_MID_tag.fasta demultiplex demultiplexing waiting for >> demultiplexing .... I can't delete it and I don't get any other >> windows to start the job. >> >> >> >> Should I re-upload it? >> >> >> >> Thanks, >> >> >> >> Alice Layton > > >

1 0

job not progressing
by Layton, Alice Camille 01 Aug '11

01 Aug '11

Dear rast, I started this job last Thursday but it seems stuck at this step G6J8P7S01.sff_no_MID_tag.fasta demultiplex demultiplexing waiting for demultiplexing .... I can't delete it and I don't get any other windows to start the job. Should I re-upload it? Thanks, Alice Layton

1 0