You are being copied on an IT ticket request. The request has been received and will be assigned to a member of our technical staff momentarily.

If you have additional information that may help us in resolving this request, please reply to this message, and your comments will be added to the ticket.

If you would like to speak to a member of the Argonne Service Desk team, call 630-252-9999, option 2, and refer to ticket 62213.

Thank you for submitting your request to the Argonne Service Desk.

Entered on 07/30/2012 at 11:06:05 CDT (GMT-0500) by Dmitriy Sklyar:
The scanning has been switched to a less aggressive schedule last week on both internal and external GSAs. They are now performing scans only during the nights and weekends with less load on the servers scanned. Please monitor your systems and let me know if you see an improvement.

Dmitriy.

Entered on 07/27/2012 at 15:54:06 CDT (GMT-0500) by Matthew Kwiatkowski:
We need to add Dana’s group to this ticket. Cyber has the same concern. The google appliances index internal sites WAY too often.

Entered on 07/27/2012 at 13:38:25 CDT (GMT-0500) by Malikah Reed:
assigning to cyber?

Entered on 07/27/2012 at 13:30:06 CDT (GMT-0500) by raffenet@mcs.anl.gov:
This scanning is still out of control. In yesterday's logs I see about
300,000 requests from each of the lab's google bots. Is there something
I should be doing to these sites to prevent this excessive scanning?

root@blizzard:/var/log/apache2# grep 146.137.96.101
blizzard.access_log.2012Jul26 | wc -l
277670
root@blizzard:/var/log/apache2# grep 130.202.84.131
blizzard.access_log.2012Jul26 | wc -l
292253

There are only about 200 pages on this entire site. Can we get support
from Google with regards to this problem? A similar issue caused me to
block the ANL google bots from www.alcf.anl.gov because it was affecting
site performance.

Ken Raffenetti
Systems Administration Associate
MCS Division - Argonne National Laboratory

On 07/10/2012 09:29 AM, Ken Raffenetti wrote:
> Google Admins,
>
> I'm seeing a pretty outrageous number of requests to
> http://collab.mcs.anl.gov. Is there any way the scanning frequency can
> be turned down on this site?
>
> In July alone, I see 1.5 million requests. This is similar to what was
> happening with www.alcf.anl.gov, which I eventually just blocked from
> being crawled by the internal Google bot.
>
> root@blizzard:/var/log/apache2# grep 146.137.96.101
> *.access_log.2012Jul* | wc -l | uniq -c
> 1 1522255
>

View Details Online: Click here to view Incident in Browser