March 2013 - Core-students - mailman3.cels.anl.gov

Core cleanup
by Ti Leggett 06 Mar '13

06 Mar '13

Today at 1pm Bill Allcock will be giving Mark Peters a walkthrough the Core for a safety focused observation. Can everyone take a minute and go through the aisles to make sure things are cleaned up?

1 0

INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=Active
by Argonne Service Desk 06 Mar '13

06 Mar '13

When replying, type your text above this line. ---------------------------------------------- Notification of Incident Change Incident: Re: excessive scanning of collab.mcs.anl.gov Incident Number: 62221 Priority: Medium Status: Active Date: 03/06/2013 Time: 10:08:04 Creation Date: 07/27/2012 Creation Time: 14:06:08 Created By: Dana Stasiak View Incident in Browser: https://tickets.anl.gov/MRcgi/MRlogin.pl?DL=62221DA12 Detail: Entered on 03/06/2013 at 10:08:04 CST (GMT-0600) by forero(a)ci.uchicago.edu: The U of C term for this is a "hull account". It's very similar to an "identity" in Userbase. The procedure is to request the hull account from Julie Irvin (jirvin(a)ci.uchicago.edu) You will need to provide the following information: Full name Date of Birth Contact email address Relation to the CI (eg who are you working with, sort of sponsorship). This will let us create a hull account. The account allows for authentication only. That means any service the person wants needs to be added individually (usually via the Grouper system). If someone needs a access to the normal suite of privileges that come with a CNet ID, then they will also contact Julie Irvin but will need to provide more information (such as SSN and resume). Let me know if you need more information. David Forero System Administrator Computation Institute University of Chicago Entered on 03/06/2013 at 09:11:05 CST (GMT-0600) by raffenet(a)mcs.anl.gov: I don't remember if I had unblocked, then blocked them again, because they were still blocked in my Apache config when I looked this morning. I'm unblocking them now and will check back on the activity later today. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 03/04/2013 05:05 PM, Argonne Service Desk wrote: > [Duplicate message snipped] Entered on 03/04/2013 at 17:05:08 CST (GMT-0600) by Dmitriy Sklyar: Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;} Ken, any info on the current state of the issue? Last we’ve talked, you were going to allow access from the GSAs and see if the changes that were maid had positive effect. Please let me know. Thank you, Dmitriy. Entered on 12/05/2012 at 10:24:34 CST (GMT-0600) by Dmitriy Sklyar: Ken has unblocked one of the servers to take at the current traffic. Further actions will be taken based on results. Entered on 10/03/2012 at 11:42:15 CDT (GMT-0500) by David Salbego: Dmitriy, He provided that information. Look at his command line cut-and-pastes. --dave Open Footprints Ticket: Assignees [Duplicate message snipped] Entered on 10/03/2012 at 11:39:12 CDT (GMT-0500) by Dmitriy Sklyar: Can you please verify which GSA server these excessive scans are coming from, http://search.anl.gov/ or http://public-search.anl.gov ? Entered on 10/03/2012 at 11:17:45 CDT (GMT-0500) by Denice DiGiacomo: Entered on 10/03/2012 at 11:03:19 CDT (GMT-0500) by [email protected]:I unblocked to 2 scanner IPs a while back for the collab.mcs.anl.gov Confluence site. Looking at the logs from yesterday, I'm still seeing over 100,000 hits per day, per bot, so no real change.root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Oct02 | wc -l133292root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Oct02 | wc -l124296Ken----- Original Message -----From: "Ken Raffenetti" <raffenet(a)mcs.anl.gov>To: help(a)anl.govSent: Thursday, September 20, 2012 9:34:14 AMSubject: Re: INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=PendingI haven't gotten around to unblocking the bots. I will try to get to ittoday and monitor for excessive scanning.Ken RaffenettiSystems Administration AssociateMCS Division - Argonne National LaboratoryOn 09/17/2012 05:13 PM, Argonne Service Desk wrote:> [Duplicate message snipped] Entered on 09/24/2012 at 17:25:47 CDT (GMT-0500) by Dana Stasiak: Thanks Ken. We'll wait to hear back from you before closing this ticket. Dana Entered on 09/20/2012 at 09:37:12 CDT (GMT-0500) by raffenet(a)mcs.anl.gov: I haven't gotten around to unblocking the bots. I will try to get to it today and monitor for excessive scanning. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 09/17/2012 05:13 PM, Argonne Service Desk wrote: > [Duplicate message snipped] Entered on 09/17/2012 at 17:13:04 CDT (GMT-0500) by Dana Stasiak: Ken, have you seen any more evidence of this behavior? Dana Entered on 07/27/2012 at 14:28:07 CDT (GMT-0500) by Dmitriy Sklyar: I have switched both internal and external GSAs to scheduled crawl and specified minimum load for Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} collab.mcs.anl.gov and Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} www.alcf.anl.gov. Please remove the blocks and let me know if the situation improves. If the blocks remain in place, no results from these sites will show up in the internal/external site search. Entered on 07/27/2012 at 14:09:19 CDT (GMT-0500) by Malikah Reed: assigning to web services Entered on 07/27/2012 at 14:06:08 CDT (GMT-0500) by Dana Stasiak: Craig, Ken, Please accept my apologies for the difficulties you have been experiencing. I have asked Dmitry to turn down the host load schedule for these servers. I am also creating a ticket to track this to completion. We have been recrawling the the collection due to some changes we are making for inside Argonne. Dana On Jul 27, 2012, at 2:32 PM, "Craig Stacey" <stace(a)mcs.anl.gov> wrote: I'd go ahead and block it until it gets figured out. -- Craig (from my mobile) Ken Raffenetti <raffenet(a)mcs.anl.gov> wrote: This scanning is still out of control. In yesterday's logs I see about 300,000 requests from each of the lab's google bots. Is there something I should be doing to these sites to prevent this excessive scanning? root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Jul26 | wc -l 277670 root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Jul26 | wc -l 292253 There are only about 200 pages on this entire site. Can we get support from Google with regards to this problem? A similar issue caused me to block the ANL google bots from www.alcf.anl.gov because it was affecting site performance. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 07/10/2012 09:29 AM, Ken Raffenetti wrote: > Google Admins, > > I'm seeing a pretty outrageous number of requests to > http://collab.mcs.anl.gov. Is there any way the scanning frequency can > be turned down on this site? > > In July alone, I see 1.5 million requests. This is similar to what was > happening with www.alcf.anl.gov, which I eventually just blocked from > being crawled by the internal Google bot. > > root@blizzard:/var/log/apache2# grep 146.137.96.101 > *.access_log.2012Jul* | wc -l | uniq -c > 1 1522255 > Current Assignees: Web Services, Dmitriy Sklyar CC(s): (permanent) raffenet(a)mcs.anl.gov, googleadmins(a)anl.gov, core-admins(a)mcs.anl.gov Customer Information: Last Name: Raffenetti First Name: Kenneth Email Address: raffenet(a)mcs.anl.gov Badge: 56518 Login ID: b56518 ANL Division: MCS Costcenter: 145 Building: 240 Room: 2135 Phone: (630) 252-3358 This is an automated message sent by the Numara FootPrints ticket tracking system, which is operated by CIS.

1 0

INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=Active
by Argonne Service Desk 06 Mar '13

06 Mar '13

When replying, type your text above this line. ---------------------------------------------- Notification of Incident Change Incident: Re: excessive scanning of collab.mcs.anl.gov Incident Number: 62221 Priority: Medium Status: Active Date: 03/06/2013 Time: 09:11:05 Creation Date: 07/27/2012 Creation Time: 14:06:08 Created By: Dana Stasiak View Incident in Browser: https://tickets.anl.gov/MRcgi/MRlogin.pl?DL=62221DA12 Detail: Entered on 03/06/2013 at 09:11:05 CST (GMT-0600) by raffenet(a)mcs.anl.gov: I don't remember if I had unblocked, then blocked them again, because they were still blocked in my Apache config when I looked this morning. I'm unblocking them now and will check back on the activity later today. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 03/04/2013 05:05 PM, Argonne Service Desk wrote: > [Duplicate message snipped] Entered on 03/04/2013 at 17:05:08 CST (GMT-0600) by Dmitriy Sklyar: Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;} Ken, any info on the current state of the issue? Last we’ve talked, you were going to allow access from the GSAs and see if the changes that were maid had positive effect. Please let me know. Thank you, Dmitriy. Entered on 12/05/2012 at 10:24:34 CST (GMT-0600) by Dmitriy Sklyar: Ken has unblocked one of the servers to take at the current traffic. Further actions will be taken based on results. Entered on 10/03/2012 at 11:42:15 CDT (GMT-0500) by David Salbego: Dmitriy, He provided that information. Look at his command line cut-and-pastes. --dave Open Footprints Ticket: Assignees [Duplicate message snipped] Entered on 10/03/2012 at 11:39:12 CDT (GMT-0500) by Dmitriy Sklyar: Can you please verify which GSA server these excessive scans are coming from, http://search.anl.gov/ or http://public-search.anl.gov ? Entered on 10/03/2012 at 11:17:45 CDT (GMT-0500) by Denice DiGiacomo: Entered on 10/03/2012 at 11:03:19 CDT (GMT-0500) by [email protected]:I unblocked to 2 scanner IPs a while back for the collab.mcs.anl.gov Confluence site. Looking at the logs from yesterday, I'm still seeing over 100,000 hits per day, per bot, so no real change.root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Oct02 | wc -l133292root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Oct02 | wc -l124296Ken----- Original Message -----From: "Ken Raffenetti" <raffenet(a)mcs.anl.gov>To: help(a)anl.govSent: Thursday, September 20, 2012 9:34:14 AMSubject: Re: INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=PendingI haven't gotten around to unblocking the bots. I will try to get to ittoday and monitor for excessive scanning.Ken RaffenettiSystems Administration AssociateMCS Division - Argonne National LaboratoryOn 09/17/2012 05:13 PM, Argonne Service Desk wrote:> [Duplicate message snipped] Entered on 09/24/2012 at 17:25:47 CDT (GMT-0500) by Dana Stasiak: Thanks Ken. We'll wait to hear back from you before closing this ticket. Dana Entered on 09/20/2012 at 09:37:12 CDT (GMT-0500) by raffenet(a)mcs.anl.gov: I haven't gotten around to unblocking the bots. I will try to get to it today and monitor for excessive scanning. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 09/17/2012 05:13 PM, Argonne Service Desk wrote: > [Duplicate message snipped] Entered on 09/17/2012 at 17:13:04 CDT (GMT-0500) by Dana Stasiak: Ken, have you seen any more evidence of this behavior? Dana Entered on 07/27/2012 at 14:28:07 CDT (GMT-0500) by Dmitriy Sklyar: I have switched both internal and external GSAs to scheduled crawl and specified minimum load for Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} collab.mcs.anl.gov and Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} www.alcf.anl.gov. Please remove the blocks and let me know if the situation improves. If the blocks remain in place, no results from these sites will show up in the internal/external site search. Entered on 07/27/2012 at 14:09:19 CDT (GMT-0500) by Malikah Reed: assigning to web services Entered on 07/27/2012 at 14:06:08 CDT (GMT-0500) by Dana Stasiak: Craig, Ken, Please accept my apologies for the difficulties you have been experiencing. I have asked Dmitry to turn down the host load schedule for these servers. I am also creating a ticket to track this to completion. We have been recrawling the the collection due to some changes we are making for inside Argonne. Dana On Jul 27, 2012, at 2:32 PM, "Craig Stacey" <stace(a)mcs.anl.gov> wrote: I'd go ahead and block it until it gets figured out. -- Craig (from my mobile) Ken Raffenetti <raffenet(a)mcs.anl.gov> wrote: This scanning is still out of control. In yesterday's logs I see about 300,000 requests from each of the lab's google bots. Is there something I should be doing to these sites to prevent this excessive scanning? root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Jul26 | wc -l 277670 root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Jul26 | wc -l 292253 There are only about 200 pages on this entire site. Can we get support from Google with regards to this problem? A similar issue caused me to block the ANL google bots from www.alcf.anl.gov because it was affecting site performance. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 07/10/2012 09:29 AM, Ken Raffenetti wrote: > Google Admins, > > I'm seeing a pretty outrageous number of requests to > http://collab.mcs.anl.gov. Is there any way the scanning frequency can > be turned down on this site? > > In July alone, I see 1.5 million requests. This is similar to what was > happening with www.alcf.anl.gov, which I eventually just blocked from > being crawled by the internal Google bot. > > root@blizzard:/var/log/apache2# grep 146.137.96.101 > *.access_log.2012Jul* | wc -l | uniq -c > 1 1522255 > Current Assignees: Web Services, Dmitriy Sklyar CC(s): (permanent) raffenet(a)mcs.anl.gov, googleadmins(a)anl.gov, core-admins(a)mcs.anl.gov Customer Information: Last Name: Raffenetti First Name: Kenneth Email Address: raffenet(a)mcs.anl.gov Badge: 56518 Login ID: b56518 ANL Division: MCS Costcenter: 145 Building: 240 Room: 2135 Phone: (630) 252-3358 This is an automated message sent by the Numara FootPrints ticket tracking system, which is operated by CIS.

2 1

Running late
by Craig Stacey 06 Mar '13

06 Mar '13

I'll be missing the mcs sync (and probably will be for the next couple of weeks) but will be in. -- Craig (from my mobile)

1 0

Snow.. you don't scare me...
by Tina Stanton-Piersanti 05 Mar '13

05 Mar '13

But due to the impending snow, Vinnie's school was cancelled. He is here with me, since Joe had training this morning (outside, hahahahahahah).. I plan on leaving after my HD shift to take Vin home and wait for all the snow to arrive so he can go out and shovel... T.

1 0

Where's Dan
by Dan Olson 05 Mar '13

05 Mar '13

I'm going to work from home today. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055

1 0

where's ken
by Ken Raffenetti 05 Mar '13

05 Mar '13

Going to work from home. While it's clear going into the lab now, the weather map looks pretty menacing. I'm still trying to get my head around the more complex features of Puppet with the hope of writing a hw/sw raid module to install the right CLI tools. -- Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory

3 2

Where's Chris?
by Chris Walker 05 Mar '13

05 Mar '13

I'll be the first one! I'm working from Userbase Central North this fine soon-to-be-wintery-day. Even the Catholics canceled school today! It's truly a special occasion, clearly a result of the leadership vacuum in Rome. I'm already settled in, I'm planning on wrapping up the sync between Userbase and Keystone, then start applying the Magellan stuff to KBase..

1 0

Where's Craig
by Craig Stacey 04 Mar '13

04 Mar '13

Going to work from home in the morning. I'm planning to work from the University in the afternoon, but that may change depending on weather. Will send a note if so. -- Craig

1 0

INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=Active
by Argonne Service Desk 04 Mar '13

04 Mar '13

When replying, type your text above this line. ---------------------------------------------- Notification of Incident Change Incident: Re: excessive scanning of collab.mcs.anl.gov Incident Number: 62221 Priority: Medium Status: Active Date: 03/04/2013 Time: 17:05:08 Creation Date: 07/27/2012 Creation Time: 14:06:08 Created By: Dana Stasiak View Incident in Browser: https://tickets.anl.gov/MRcgi/MRlogin.pl?DL=62221DA12 Detail: Entered on 03/04/2013 at 17:05:08 CST (GMT-0600) by Dmitriy Sklyar: Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;} Ken, any info on the current state of the issue? Last we’ve talked, you were going to allow access from the GSAs and see if the changes that were maid had positive effect. Please let me know. Thank you, Dmitriy. Entered on 12/05/2012 at 10:24:34 CST (GMT-0600) by Dmitriy Sklyar: Ken has unblocked one of the servers to take at the current traffic. Further actions will be taken based on results. Entered on 10/03/2012 at 11:42:15 CDT (GMT-0500) by David Salbego: Dmitriy, He provided that information. Look at his command line cut-and-pastes. --dave Open Footprints Ticket: Assignees [Duplicate message snipped] Entered on 10/03/2012 at 11:39:12 CDT (GMT-0500) by Dmitriy Sklyar: Can you please verify which GSA server these excessive scans are coming from, http://search.anl.gov/ or http://public-search.anl.gov ? Entered on 10/03/2012 at 11:17:45 CDT (GMT-0500) by Denice DiGiacomo: Entered on 10/03/2012 at 11:03:19 CDT (GMT-0500) by [email protected]:I unblocked to 2 scanner IPs a while back for the collab.mcs.anl.gov Confluence site. Looking at the logs from yesterday, I'm still seeing over 100,000 hits per day, per bot, so no real change.root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Oct02 | wc -l133292root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Oct02 | wc -l124296Ken----- Original Message -----From: "Ken Raffenetti" <raffenet(a)mcs.anl.gov>To: help(a)anl.govSent: Thursday, September 20, 2012 9:34:14 AMSubject: Re: INFO: Re: excessive scanning of collab.mcs.anl.gov, Ticket No. 62221, Status=PendingI haven't gotten around to unblocking the bots. I will try to get to ittoday and monitor for excessive scanning.Ken RaffenettiSystems Administration AssociateMCS Division - Argonne National LaboratoryOn 09/17/2012 05:13 PM, Argonne Service Desk wrote:> [Duplicate message snipped] Entered on 09/24/2012 at 17:25:47 CDT (GMT-0500) by Dana Stasiak: Thanks Ken. We'll wait to hear back from you before closing this ticket. Dana Entered on 09/20/2012 at 09:37:12 CDT (GMT-0500) by raffenet(a)mcs.anl.gov: I haven't gotten around to unblocking the bots. I will try to get to it today and monitor for excessive scanning. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 09/17/2012 05:13 PM, Argonne Service Desk wrote: > [Duplicate message snipped] Entered on 09/17/2012 at 17:13:04 CDT (GMT-0500) by Dana Stasiak: Ken, have you seen any more evidence of this behavior? Dana Entered on 07/27/2012 at 14:28:07 CDT (GMT-0500) by Dmitriy Sklyar: I have switched both internal and external GSAs to scheduled crawl and specified minimum load for Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} collab.mcs.anl.gov and Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";} www.alcf.anl.gov. Please remove the blocks and let me know if the situation improves. If the blocks remain in place, no results from these sites will show up in the internal/external site search. Entered on 07/27/2012 at 14:09:19 CDT (GMT-0500) by Malikah Reed: assigning to web services Entered on 07/27/2012 at 14:06:08 CDT (GMT-0500) by Dana Stasiak: Craig, Ken, Please accept my apologies for the difficulties you have been experiencing. I have asked Dmitry to turn down the host load schedule for these servers. I am also creating a ticket to track this to completion. We have been recrawling the the collection due to some changes we are making for inside Argonne. Dana On Jul 27, 2012, at 2:32 PM, "Craig Stacey" <stace(a)mcs.anl.gov> wrote: I'd go ahead and block it until it gets figured out. -- Craig (from my mobile) Ken Raffenetti <raffenet(a)mcs.anl.gov> wrote: This scanning is still out of control. In yesterday's logs I see about 300,000 requests from each of the lab's google bots. Is there something I should be doing to these sites to prevent this excessive scanning? root@blizzard:/var/log/apache2# grep 146.137.96.101 blizzard.access_log.2012Jul26 | wc -l 277670 root@blizzard:/var/log/apache2# grep 130.202.84.131 blizzard.access_log.2012Jul26 | wc -l 292253 There are only about 200 pages on this entire site. Can we get support from Google with regards to this problem? A similar issue caused me to block the ANL google bots from www.alcf.anl.gov because it was affecting site performance. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 07/10/2012 09:29 AM, Ken Raffenetti wrote: > Google Admins, > > I'm seeing a pretty outrageous number of requests to > http://collab.mcs.anl.gov. Is there any way the scanning frequency can > be turned down on this site? > > In July alone, I see 1.5 million requests. This is similar to what was > happening with www.alcf.anl.gov, which I eventually just blocked from > being crawled by the internal Google bot. > > root@blizzard:/var/log/apache2# grep 146.137.96.101 > *.access_log.2012Jul* | wc -l | uniq -c > 1 1522255 > Current Assignees: Web Services, Dmitriy Sklyar CC(s): (permanent) raffenet(a)mcs.anl.gov, googleadmins(a)anl.gov, core-admins(a)mcs.anl.gov Customer Information: Last Name: Raffenetti First Name: Kenneth Email Address: raffenet(a)mcs.anl.gov Badge: 56518 Login ID: b56518 ANL Division: MCS Costcenter: 145 Building: 240 Room: 2135 Phone: (630) 252-3358 This is an automated message sent by the Numara FootPrints ticket tracking system, which is operated by CIS.

1 0