Not really. There is a query size database limitation when accessing data from workbench md5 sets. There is a SQL statment of 'select <stuff> from <job_table> where md5 in (list of workbench md5s)' going on at the backend. Once the list is too large the time taken becomes too long and we get into webserver timeouts. 50,000 md5s is too much. It kind of depends on the size of the data table, but we start getting timeouts somewhere between 20-40,000 md5s. We could split up the query and get the results in parallel, but we would still get a timeout at some point as data gets larger. -Travis On Tue, Jul 19, 2011 at 12:06 PM, Mark D'Souza <[email protected]> wrote:
Hi Travis, any hope for this guy? Mark
Begin forwarded message:
From: Mark D'Souza <[email protected]> Date: July 19, 2011 12:05:00 PM CDT To: [email protected] Cc: [email protected] Subject: Re: [mg-rast] bug for tree generation
Hi,
The database query required to create the trees becomes more complex with number of metagenomes and increasing number of filtering steps causing lengthy times for retrieval of data and may cause the tree visualization to fail, we will check and see what can be done.
Regards, Mark D'Souza -- for the MG-RAST team
[email protected] http://metagenomics.anl.gov/
If you need to respond to this email please address it to the mailing-list [email protected]
On Jul 19, 2011, at 9:03 AM, [email protected] wrote:
Hello MG-RAST team, I wonder if you could give me a help. I can't visualize a specific data set saved at workbench as a tree scheme. I picked metagenomes: 4461456.3, 4461455.3, 4461454.3, 4461453.3,4461452.3, 4461451.3, 4453374.3, 4461444.3, and pooled the Bacteria domain related proteins (total of 48557 unique proteins, using GenBank and cutoff of 1e-5 and Min Alig Length of 50bp). When I try to generate a tree using this set of ptns the "data generation progress" keep going forever. When I use all ptns info from this group of metagemones, a shorter workbench set of ptns or less metagenomes I could get the tree scheme tough. So, it seems to be a bug that happens specificaly with the set of ptns that I chose. Thanks, Amaro