I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off. Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine. I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly. As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs. Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb. May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile) Dan Olson <[email protected]> wrote: I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off. Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine. I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly. As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs. Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb. May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
Should wind be up? It's not pingable right now. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
It should be up. The root volume spec in grub is probably wrong. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
squall (www.alcf) move is complete to vserver1-221. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues It should be up. The root volume spec in grub is probably wrong. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
I think I've got the grant issues for wind figured out. All those extra gateway entries in /etc/network/interfaces were affecting how the mysql server saw connections. Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory On 05/27/2012 01:17 PM, Dan Olson wrote:
squall (www.alcf) move is complete to vserver1-221.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues
It should be up. The root volume spec in grub is probably wrong.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
Thanks a lot. So the status of things as I understand it is: ALCF website moved to new vserver. franklin/owney move is in progress, happening live status.alcf, blizzard, and dust are all staying on the server with the memory error for now. Yes? On May 27, 2012, at 1:36 PM, Ken Raffenetti wrote:
I think I've got the grant issues for wind figured out. All those extra gateway entries in /etc/network/interfaces were affecting how the mysql server saw connections.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 01:17 PM, Dan Olson wrote:
squall (www.alcf) move is complete to vserver1-221.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues
It should be up. The root volume spec in grub is probably wrong.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
-- Craig
Yes ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Craig Stacey" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: "Dan Olson" <[email protected]>, [email protected] Sent: Sunday, May 27, 2012 1:38:00 PM Subject: Re: 221 vserver issues Thanks a lot. So the status of things as I understand it is: ALCF website moved to new vserver. franklin/owney move is in progress, happening live status.alcf, blizzard, and dust are all staying on the server with the memory error for now. Yes? On May 27, 2012, at 1:36 PM, Ken Raffenetti wrote:
I think I've got the grant issues for wind figured out. All those extra gateway entries in /etc/network/interfaces were affecting how the mysql server saw connections.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 01:17 PM, Dan Olson wrote:
squall (www.alcf) move is complete to vserver1-221.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues
It should be up. The root volume spec in grub is probably wrong.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
-- Craig
franklin is currently at 42% transferred to vserver4-221. It should complete around 5:30 or 6pm. Hopefully that goes well, if it doesn't vserver3-221 will leave the source VM paused, we can unpause it if necessary. I verified rt, squall, and wind (wikis) are using coredb, not coredb-2. I think its a good time to walk away for a bit. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Craig Stacey" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:41:09 PM Subject: Re: 221 vserver issues Yes ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Craig Stacey" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: "Dan Olson" <[email protected]>, [email protected] Sent: Sunday, May 27, 2012 1:38:00 PM Subject: Re: 221 vserver issues Thanks a lot. So the status of things as I understand it is: ALCF website moved to new vserver. franklin/owney move is in progress, happening live status.alcf, blizzard, and dust are all staying on the server with the memory error for now. Yes? On May 27, 2012, at 1:36 PM, Ken Raffenetti wrote:
I think I've got the grant issues for wind figured out. All those extra gateway entries in /etc/network/interfaces were affecting how the mysql server saw connections.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 01:17 PM, Dan Olson wrote:
squall (www.alcf) move is complete to vserver1-221.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues
It should be up. The root volume spec in grub is probably wrong.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
-- Craig
Agreed. And thanks again. -- Craig On May 27, 2012, at 1:58 PM, Dan Olson <[email protected]> wrote:
franklin is currently at 42% transferred to vserver4-221. It should complete around 5:30 or 6pm. Hopefully that goes well, if it doesn't vserver3-221 will leave the source VM paused, we can unpause it if necessary.
I verified rt, squall, and wind (wikis) are using coredb, not coredb-2. I think its a good time to walk away for a bit.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Craig Stacey" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:41:09 PM Subject: Re: 221 vserver issues
Yes
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Craig Stacey" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: "Dan Olson" <[email protected]>, [email protected] Sent: Sunday, May 27, 2012 1:38:00 PM Subject: Re: 221 vserver issues
Thanks a lot. So the status of things as I understand it is:
ALCF website moved to new vserver. franklin/owney move is in progress, happening live
status.alcf, blizzard, and dust are all staying on the server with the memory error for now.
Yes?
On May 27, 2012, at 1:36 PM, Ken Raffenetti wrote:
I think I've got the grant issues for wind figured out. All those extra gateway entries in /etc/network/interfaces were affecting how the mysql server saw connections.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 01:17 PM, Dan Olson wrote:
squall (www.alcf) move is complete to vserver1-221.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Dan Olson" <[email protected]> To: "Ken Raffenetti" <[email protected]> Cc: [email protected] Sent: Sunday, May 27, 2012 1:05:19 PM Subject: Re: 221 vserver issues
It should be up. The root volume spec in grub is probably wrong.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
----- Original Message ----- From: "Ken Raffenetti" <[email protected]> To: [email protected] Sent: Sunday, May 27, 2012 12:59:37 PM Subject: Re: 221 vserver issues
I'm kicking it. The console showed it in busybox. If I shouldn't be doing that, please let me know.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:49 PM, Ken Raffenetti wrote:
Should wind be up? It's not pingable right now.
Ken Raffenetti Systems Administration Associate MCS Division - Argonne National Laboratory
On 05/27/2012 12:43 PM, Craig Stacey wrote:
Thanks. I'm not sure how much help I can be but I can take a look at coredb in a little bit. -- Craig (from my mobile)
Dan Olson <[email protected]> wrote:
I've spent the last couple hours trying to find new homes for the critical vms that are on vserver3-221. Memory mirroring seems to have made the problem worse, instead of just panicing the box, it would reboot after a bit with no log info. So I turned that off.
Moving the VMs live also failed, I attempted to move status.alcf to vserver1-221, which panicd that machine.
I'm currently trying to move franklin to vserver4-221, and its progressing, but slowly.
As far as I can tell the machines that are critical are squall (www.alcf) - move in progress, currently offline franklin (lists) - hot migration still progressing to vserver4-221, we will see how this pans out. If we need to migrate it offline it will take 5.5 hrs.
Needs work - if anyone wants to assist - coredb-2, something is wrong with the grants for the wikis on coredb, when I point wind at coredb the wikis don't work. Other than this we should be able to point everything at coredb.
May stay on the troubled vserver, we will try to get them off from the top to the bottom blizzard (not sure how important confluence is at this point) status.alcf dust
We do have an additional chassis that is not being used by alcf that matches vserver3-221 labeled vserver2-alcf in building 240, however swapping the disks out of vserver3-221 into that machine seems more risky than the VM migration. It would be nice to have that machine moved into place in 221 just in case it comes to that.
---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055
-- Craig
participants (3)
-
Craig Stacey -
Dan Olson -
Ken Raffenetti