FYI: Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing. -corby
I took a look at the processes on wind and found that there were 464 apache2 processes, when I did an /etc/init.d/apache2 stop, there were 456 remaining. I ran pkill apache2 which got rid of almost all of them (kill -9 on the rest) and then restarted apache2 which gave me 8 new processes. Wind now has under 250 processes, and all the web-sites are responding again. In addition the load average dropped to ~0. -corby On Jan 30, 2009, at 7:38 AM, Schmitz Corby wrote:
FYI:
Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing.
-corby
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This sounds like a scan, whether it is a haxor or google, we should find out and possibly add in some .htaccess foo to keep this from continuing. Rick Schmitz Corby wrote:
I took a look at the processes on wind and found that there were 464 apache2 processes, when I did an /etc/init.d/apache2 stop, there were 456 remaining. I ran pkill apache2 which got rid of almost all of them (kill -9 on the rest) and then restarted apache2 which gave me 8 new processes. Wind now has under 250 processes, and all the web-sites are responding again. In addition the load average dropped to ~0.
-corby
On Jan 30, 2009, at 7:38 AM, Schmitz Corby wrote:
FYI:
Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing.
-corby
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJgx6ddavihOKkCCERAtxYAJsGrTZ64/XwVtzNtHK6XXRb+hiY4ACeI9KM WXEfZTqcCtZ0+xQAdEprmVQ= =j9eI -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I think this may have been me moving some sites but not creating their log directories. There were many errors trying to write to and rotate those log files which may have caused a bunch of processes to go zombie. Ken Rick Bradshaw wrote:
This sounds like a scan, whether it is a haxor or google, we should find out and possibly add in some .htaccess foo to keep this from continuing.
Rick
Schmitz Corby wrote:
I took a look at the processes on wind and found that there were 464 apache2 processes, when I did an /etc/init.d/apache2 stop, there were 456 remaining. I ran pkill apache2 which got rid of almost all of them (kill -9 on the rest) and then restarted apache2 which gave me 8 new processes. Wind now has under 250 processes, and all the web-sites are responding again. In addition the load average dropped to ~0.
-corby
On Jan 30, 2009, at 7:38 AM, Schmitz Corby wrote:
FYI:
Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing.
-corby
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmHcPcACgkQvRJB9CBtvwiHmgCfQTInbRAzuQTP/Yaq2oD5CztO 5YMAnje9VGFezmaPOTER21k4obwUGp3i =30WT -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Could this have affected wikis on wind? I had to restart Apache on Saturday to get the wikis back online. == Craig On Feb 2, 2009, at 4:17 PM, Kenneth Raffenetti wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I think this may have been me moving some sites but not creating their log directories. There were many errors trying to write to and rotate those log files which may have caused a bunch of processes to go zombie.
Ken
Rick Bradshaw wrote:
This sounds like a scan, whether it is a haxor or google, we should find out and possibly add in some .htaccess foo to keep this from continuing.
Rick
Schmitz Corby wrote:
I took a look at the processes on wind and found that there were 464 apache2 processes, when I did an /etc/init.d/apache2 stop, there were 456 remaining. I ran pkill apache2 which got rid of almost all of them (kill -9 on the rest) and then restarted apache2 which gave me 8 new processes. Wind now has under 250 processes, and all the web-sites are responding again. In addition the load average dropped to ~0.
-corby
On Jan 30, 2009, at 7:38 AM, Schmitz Corby wrote:
FYI:
Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing.
-corby
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkmHcPcACgkQvRJB9CBtvwiHmgCfQTInbRAzuQTP/Yaq2oD5CztO 5YMAnje9VGFezmaPOTER21k4obwUGp3i =30WT -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkmHcpsACgkQbVNGA3HOzQvJkgCcCgcR6aVBnMLWxpDsZkQxzq6A tXUAnjl5EtVoV9Lf4poQJh5AMgwa6ojK =bIWC -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Yeah, wind got a couple vhosts from bravo last week. I created a handful of directories this morning, then killed off all apache processes and restarted. The process count has stayed low throughout the day. Hopefully it stays there. Ken Craig Stacey wrote:
Could this have affected wikis on wind? I had to restart Apache on Saturday to get the wikis back online. == Craig
On Feb 2, 2009, at 4:17 PM, Kenneth Raffenetti wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I think this may have been me moving some sites but not creating their log directories. There were many errors trying to write to and rotate those log files which may have caused a bunch of processes to go zombie.
Ken
Rick Bradshaw wrote:
This sounds like a scan, whether it is a haxor or google, we should find out and possibly add in some .htaccess foo to keep this from continuing.
Rick
Schmitz Corby wrote:
I took a look at the processes on wind and found that there were 464 apache2 processes, when I did an /etc/init.d/apache2 stop, there were 456 remaining. I ran pkill apache2 which got rid of almost all of them (kill -9 on the rest) and then restarted apache2 which gave me 8 new processes. Wind now has under 250 processes, and all the web-sites are responding again. In addition the load average dropped to ~0.
-corby
On Jan 30, 2009, at 7:38 AM, Schmitz Corby wrote:
FYI:
Web sites hosted on breeze and wind went offline ~0215hrs this morning (thats my best estimate right now). I restarted apache on both and the alarms are clearing.
-corby
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmHc4IACgkQvRJB9CBtvwh3CgCfQR9SiXwypTW2AHo1wcvSeqBg AdUAn0Ys0MdXYmbva1XkDsOMXDiqWz6q =ambF -----END PGP SIGNATURE-----
participants (4)
-
Craig Stacey -
Kenneth Raffenetti -
Rick Bradshaw -
Schmitz Corby