Petsc will go down. Enterprise is going to go down. Consolo going down. Newton will go down. Cookie is going down. cg might, it has 3 PSs, but I don’t know if it’s N+1 or what ... -- Adam Max Trefonides CELS Systems [email protected]
On May 5, 2015, at 3:45 PM, Leggett, Torrance I. <[email protected]> wrote:
FYI, the racks labeled MCS are all of systems row. Is everything in there dual fed or do we know what this might take out?
From: <Leggett>, Ti Leggett <[email protected]> Date: Tuesday, May 5, 2015 at 3:17 PM To: CELS Admins <[email protected]> Subject: [CELS Sysadmins] FW: chameleon power
Unfortunately what I thought were all datacenter admins being included in this discussion were only the CIS datacenter admins. Right now the racks listed below are slated to lose the UPS-PDU-2 feed tomorrow at 8am. If this is unacceptable, I need to know ASAP to get this re-scheduled.
From: <Bathje>, "Frederick M." <[email protected]> Date: Tuesday, April 21, 2015 at 1:57 PM To: Ti Leggett <[email protected]>, "Stacey, Craig J." <[email protected]>, "Roberts, John E." <[email protected]> Cc: "[email protected]" <[email protected]> Subject: RE: chameleon power
They are bringing their contractor out again to quote the additional circuit. With the impending shutdown, do we want to try and coordinate this work to happen during the shutdown instead of taking 2 outages? Can this wait that long? I’ve listed what I have documented as powered off of UPS-PDU-2 below if we need to schedule it sooner.
Thanks, Fred
BB-57 MCS BC-57 MCS BD-57 MCS BE-57 MCS BF-57 MCS BH-53 EVS BG-52 EVS BF-52 EVS AZ-53 I2U2 AX-53 I2U2 AQ-49 breadboard BF-49 pads BG-49 pads BT-36 LCF BT-41 LCF CF-41 LCF CF-49 LCF
From: Leggett, Torrance I. Sent: Friday, April 17, 2015 4:01 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: [email protected]; Winkler, Linda Subject: Re: chameleon power
After going through the final hardware verification with Dell they're now saying the management rack has 2 L21-30's. The racks will also be laid out compute-management-compute in floor layout
So I'll need
(3) L21-30 Compute rack 1 (2) L21-30 Management (3) L21-30 Compute rack 2
From: <Bathje>, "Frederick M." <[email protected]> Date: Tuesday, April 14, 2015 at 7:49 AM To: Ti Leggett <[email protected]>, "Stacey, Craig J." <[email protected]>, "Roberts, John E." <[email protected]> Cc: "[email protected]" <[email protected]>, "Winkler, Linda" <[email protected]> Subject: RE: chameleon power
I’ve reconfigured the diagram for that rack size. We can do 3 grate tiles, but it may end up being overkill. Total for all 3 racks would be 33KW. 2 grate tiles and 1 regular perforated tile would put it around 31KW, and there is still excess cooling from other surrounding tiles. We can adjust if necessary. ALCF has excess Tate GrateAir tiles that Bill said we can use. I’ve put in an ANGUS tickets to get a quote for the circuit runs. Let me know if you hear from Dell on the rack layout.
Fred
From: Leggett, Torrance I. Sent: Friday, April 10, 2015 3:14 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: [email protected]; Winkler, Linda Subject: Re: chameleon power
Based on how these racks are provisioned, my tentative answer is that a compute on either side would be fine, though I will *try* to get a confirmation from Dell. I think the cabling between the racks is a network cable from each compute to the management aggregate switch, so the normal size cut should be more than enough to handle the power whip and the fiber. It's acceptable to put them on a single PDU since there's no dual feed for the management rack we run the risk of the entire machine going down if it does so might as well guarantee it to. The racks are all APC 48U racks: http://accessories.dell.com/sna/productdetail.aspx?c=us&l=en&s=bsd&cs=04&sku....
Do you mean to have 3 grate tiles since you mention excess from the management?
From: <Bathje>, "Frederick M." <[email protected]> Date: Friday, April 10, 2015 at 2:59 PM To: Ti Leggett <[email protected]>, "Stacey, Craig J." <[email protected]>, "Roberts, John E." <[email protected]> Cc: "[email protected]" <[email protected]>, "Winkler, Linda" <[email protected]> Subject: RE: chameleon power
Ti,
Since cabling has already been ordered, we can keep them where you had originally planned them. When we add racks in the future, we can just build inward. Based on all the documentation you’ve provided, it looks like you need:
· (3) L21-30 receptacles for Compute rack 1 · (3) L21-30 receptacles for Compute rack 2 · (1) L21-30 receptacle for management rack · (2) Grate tiles to cool the compute racks o These can cool around 12KW each, but with how much we overcool and the excess capacity from the management rack tile there should not be an issue. I’ll have to check if we have these or need to order them. · (3) cut floor tiles for cdu cord and network cabling. Do you know what size cutout you will need?
I will put in an ANGUS ticket to let the building know what we need, and they will request that we walk it down with them. We will also need to coordinate a shutdown of the PDU that we will install circuits on. UPS-PDU-2 is near your requested location and has the lowest load of all the UPS PDUs. Is it acceptable to put all of the circuits on one PDU?
Can you tell me what the rack size will be? I’ve attached a diagram using 30”x48”, but we may want to change the receptacle locations based on the actual size. It would be best if we can put the management rack in the center, is that how you had planned it?
Thanks, Fred
From: Leggett, Torrance I. Sent: Friday, April 10, 2015 1:42 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: [email protected]; Winkler, Linda Subject: Re: chameleon power
I can't get anything out of Dell, so I'm going with my gut that this isn't dual fed because of the odd numbers of CDUs. We'll proceed with a single fed assumption for these racks.
Also, Is there a plan to move the racks from across from FutureGrid? If so, Linda needs to know since she had already ordered and has received the appropriate length cables when she was told it was going to be in front of FutureGrid.
From: <Bathje>, "Frederick M." <[email protected]> Date: Friday, March 27, 2015 at 10:23 AM To: Ti Leggett <[email protected]>, "Stacey, Craig J." <[email protected]>, "Roberts, John E." <[email protected]> Cc: "[email protected]" <[email protected]> Subject: RE: chameleon power
Thanks, Ti.
Let me know when you get confirmation on the redundant power supplies.
Fred
From: Leggett, Torrance I. Sent: Wednesday, March 25, 2015 8:39 AM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: [email protected] Subject: Re: chameleon power
After much pulling of teeth, I finally got some specifics out of Dell about the per-rack power requirements. Chameleon will have 2 compute racks (containing 54 devices each) and 1 management rack. Each compute rack will have 3 CDUs (http://accessories.dell.com/sna/productdetail.aspx?c=us&l=en&s=bsd&cs=04&sku...). Each compute rack will have a peak of 15kW draw. The management rack will have one of those CDUs with a peak draw of 3kW. I'm still waiting for final confirmation from Dell about dual-feed requirements, but since there are only 3 CDUs in each rack I can't see how it could be designed for dual-feed where all devices survive one leg going down, so I would assume single feed power.
From: <Bathje>, "Frederick M." <[email protected]> Date: Tuesday, March 24, 2015 at 3:43 PM To: Ti Leggett <[email protected]>, "Stacey, Craig J." <[email protected]>, "Roberts, John E." <[email protected]> Cc: "[email protected]" <[email protected]> Subject: RE: chameleon power
I took a look and there are no L21-30R outlets in that area. Mostly pin and sleeve 60 amp circuits off PDU3 and a few L6-30.
UPS PDU-2 looks to be a good option for where to install the circuits based on load. Once you get the power information for the management rack, I can have building management quote a circuit install. We will need to coordinate a shutdown of the panel that we will be adding them on.
From a space perspective Do you know what make and model rack you will be using? AY-41, AZ-41, BA-41, and BB-41 would make the most sense cooling wise.
Thanks, Fred
From: Leggett, Torrance I. Sent: Monday, March 23, 2015 2:38 PM To: Bathje, Frederick M.; Stacey, Craig J.; Leggett, Torrance I. Subject: Re: chameleon power
I'm getting clarification from Dell. I'll let you know what hear back from them.
From: <Bathje>, "Frederick M." <[email protected]> Date: Monday, March 23, 2015 at 1:22 PM To: "Stacey, Craig J." <[email protected]>, Ti Leggett <[email protected]> Subject: chameleon power
Hi Craig/Ti,
From the email it looks like the 2 cluster racks will need (2) non redundant L21-30R fed from UPS (4 total circuits). Do we know what the management rack will need?
Regarding deploying power for Chameleon, I remembered something Craig said a while back when planning for the “bio move” with Rocky. I think you said you had unused power whips somewhere else that Rocky could have used (Rene ended up putting his equipment in CIS racks)? Do you know if this is still true, and something that would make sense to use for this project?
Thanks, Fred