May 2015 - Core-students - mailman3.cels.anl.gov

[CELS Sysadmins] FW: chameleon power
by Leggett, Torrance I. 05 May '15

05 May '15

Unfortunately what I thought were all datacenter admins being included in this discussion were only the CIS datacenter admins. Right now the racks listed below are slated to lose the UPS-PDU-2 feed tomorrow at 8am. If this is unacceptable, I need to know ASAP to get this re-scheduled. From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Tuesday, April 21, 2015 at 1:57 PM To: Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>>, "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, "Roberts, John E." <jeroberts(a)anl.gov<mailto:[email protected]>> Cc: "cis-datacenter-admins(a)anl.gov<mailto:[email protected]>" <cis-datacenter-admins(a)anl.gov<mailto:[email protected]>> Subject: RE: chameleon power They are bringing their contractor out again to quote the additional circuit. With the impending shutdown, do we want to try and coordinate this work to happen during the shutdown instead of taking 2 outages? Can this wait that long? I’ve listed what I have documented as powered off of UPS-PDU-2 below if we need to schedule it sooner. Thanks, Fred BB-57 MCS BC-57 MCS BD-57 MCS BE-57 MCS BF-57 MCS BH-53 EVS BG-52 EVS BF-52 EVS AZ-53 I2U2 AX-53 I2U2 AQ-49 breadboard BF-49 pads BG-49 pads BT-36 LCF BT-41 LCF CF-41 LCF CF-49 LCF From: Leggett, Torrance I. Sent: Friday, April 17, 2015 4:01 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: cis-datacenter-admins(a)anl.gov<mailto:[email protected]>; Winkler, Linda Subject: Re: chameleon power After going through the final hardware verification with Dell they're now saying the management rack has 2 L21-30's. The racks will also be laid out compute-management-compute in floor layout So I'll need (3) L21-30 Compute rack 1 (2) L21-30 Management (3) L21-30 Compute rack 2 From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Tuesday, April 14, 2015 at 7:49 AM To: Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>>, "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, "Roberts, John E." <jeroberts(a)anl.gov<mailto:[email protected]>> Cc: "cis-datacenter-admins(a)anl.gov<mailto:[email protected]>" <cis-datacenter-admins(a)anl.gov<mailto:[email protected]>>, "Winkler, Linda" <winkler(a)mcs.anl.gov<mailto:[email protected]>> Subject: RE: chameleon power I’ve reconfigured the diagram for that rack size. We can do 3 grate tiles, but it may end up being overkill. Total for all 3 racks would be 33KW. 2 grate tiles and 1 regular perforated tile would put it around 31KW, and there is still excess cooling from other surrounding tiles. We can adjust if necessary. ALCF has excess Tate GrateAir tiles that Bill said we can use. I’ve put in an ANGUS tickets to get a quote for the circuit runs. Let me know if you hear from Dell on the rack layout. Fred From: Leggett, Torrance I. Sent: Friday, April 10, 2015 3:14 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: cis-datacenter-admins(a)anl.gov<mailto:[email protected]>; Winkler, Linda Subject: Re: chameleon power Based on how these racks are provisioned, my tentative answer is that a compute on either side would be fine, though I will *try* to get a confirmation from Dell. I think the cabling between the racks is a network cable from each compute to the management aggregate switch, so the normal size cut should be more than enough to handle the power whip and the fiber. It's acceptable to put them on a single PDU since there's no dual feed for the management rack we run the risk of the entire machine going down if it does so might as well guarantee it to. The racks are all APC 48U racks: http://accessories.dell.com/sna/productdetail.aspx?c=us&l=en&s=bsd&cs=04&sk…. Do you mean to have 3 grate tiles since you mention excess from the management? From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Friday, April 10, 2015 at 2:59 PM To: Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>>, "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, "Roberts, John E." <jeroberts(a)anl.gov<mailto:[email protected]>> Cc: "cis-datacenter-admins(a)anl.gov<mailto:[email protected]>" <cis-datacenter-admins(a)anl.gov<mailto:[email protected]>>, "Winkler, Linda" <winkler(a)mcs.anl.gov<mailto:[email protected]>> Subject: RE: chameleon power Ti, Since cabling has already been ordered, we can keep them where you had originally planned them. When we add racks in the future, we can just build inward. Based on all the documentation you’ve provided, it looks like you need: · (3) L21-30 receptacles for Compute rack 1 · (3) L21-30 receptacles for Compute rack 2 · (1) L21-30 receptacle for management rack · (2) Grate tiles to cool the compute racks o These can cool around 12KW each, but with how much we overcool and the excess capacity from the management rack tile there should not be an issue. I’ll have to check if we have these or need to order them. · (3) cut floor tiles for cdu cord and network cabling. Do you know what size cutout you will need? I will put in an ANGUS ticket to let the building know what we need, and they will request that we walk it down with them. We will also need to coordinate a shutdown of the PDU that we will install circuits on. UPS-PDU-2 is near your requested location and has the lowest load of all the UPS PDUs. Is it acceptable to put all of the circuits on one PDU? Can you tell me what the rack size will be? I’ve attached a diagram using 30”x48”, but we may want to change the receptacle locations based on the actual size. It would be best if we can put the management rack in the center, is that how you had planned it? Thanks, Fred From: Leggett, Torrance I. Sent: Friday, April 10, 2015 1:42 PM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: cis-datacenter-admins(a)anl.gov<mailto:[email protected]>; Winkler, Linda Subject: Re: chameleon power I can't get anything out of Dell, so I'm going with my gut that this isn't dual fed because of the odd numbers of CDUs. We'll proceed with a single fed assumption for these racks. Also, Is there a plan to move the racks from across from FutureGrid? If so, Linda needs to know since she had already ordered and has received the appropriate length cables when she was told it was going to be in front of FutureGrid. From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Friday, March 27, 2015 at 10:23 AM To: Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>>, "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, "Roberts, John E." <jeroberts(a)anl.gov<mailto:[email protected]>> Cc: "cis-datacenter-admins(a)anl.gov<mailto:[email protected]>" <cis-datacenter-admins(a)anl.gov<mailto:[email protected]>> Subject: RE: chameleon power Thanks, Ti. Let me know when you get confirmation on the redundant power supplies. Fred From: Leggett, Torrance I. Sent: Wednesday, March 25, 2015 8:39 AM To: Bathje, Frederick M.; Leggett, Torrance I.; Stacey, Craig J.; Roberts, John E. Cc: cis-datacenter-admins(a)anl.gov<mailto:[email protected]> Subject: Re: chameleon power After much pulling of teeth, I finally got some specifics out of Dell about the per-rack power requirements. Chameleon will have 2 compute racks (containing 54 devices each) and 1 management rack. Each compute rack will have 3 CDUs (http://accessories.dell.com/sna/productdetail.aspx?c=us&l=en&s=bsd&cs=04&sk…) Each compute rack will have a peak of 15kW draw. The management rack will have one of those CDUs with a peak draw of 3kW. I'm still waiting for final confirmation from Dell about dual-feed requirements, but since there are only 3 CDUs in each rack I can't see how it could be designed for dual-feed where all devices survive one leg going down, so I would assume single feed power. From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Tuesday, March 24, 2015 at 3:43 PM To: Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>>, "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, "Roberts, John E." <jeroberts(a)anl.gov<mailto:[email protected]>> Cc: "cis-datacenter-admins(a)anl.gov<mailto:[email protected]>" <cis-datacenter-admins(a)anl.gov<mailto:[email protected]>> Subject: RE: chameleon power I took a look and there are no L21-30R outlets in that area. Mostly pin and sleeve 60 amp circuits off PDU3 and a few L6-30. UPS PDU-2 looks to be a good option for where to install the circuits based on load. Once you get the power information for the management rack, I can have building management quote a circuit install. We will need to coordinate a shutdown of the panel that we will be adding them on. >From a space perspective Do you know what make and model rack you will be using? AY-41, AZ-41, BA-41, and BB-41 would make the most sense cooling wise. Thanks, Fred From: Leggett, Torrance I. Sent: Monday, March 23, 2015 2:38 PM To: Bathje, Frederick M.; Stacey, Craig J.; Leggett, Torrance I. Subject: Re: chameleon power I'm getting clarification from Dell. I'll let you know what hear back from them. From: <Bathje>, "Frederick M." <fbathje(a)anl.gov<mailto:[email protected]>> Date: Monday, March 23, 2015 at 1:22 PM To: "Stacey, Craig J." <stace(a)anl.gov<mailto:[email protected]>>, Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>> Subject: chameleon power Hi Craig/Ti, >From the email it looks like the 2 cluster racks will need (2) non redundant L21-30R fed from UPS (4 total circuits). Do we know what the management rack will need? Regarding deploying power for Chameleon, I remembered something Craig said a while back when planning for the “bio move” with Rocky. I think you said you had unused power whips somewhere else that Rocky could have used (Rene ended up putting his equipment in CIS racks)? Do you know if this is still true, and something that would make sense to use for this project? Thanks, Fred

1 0

Re: [CELS Sysadmins] Invalid nagios config somewhere?
by Roberts, John E. 05 May '15

05 May '15

I see this. I should only be active on auth and auth2 define command{ command_name check_crypto_radius command_line $USER1$/check_radius -H $HOSTADDRESS$ -u nagios-radius-test-user -p q1w2e3r4 -F /etc/radiusclient/radiusclient.conf -P 1812 -t 60 } -- John Roberts Argonne National Laboratory CELS Systems jeroberts(a)anl.gov From: <Leggett>, "Torrance I." <leggett(a)mcs.anl.gov<mailto:[email protected]>> Date: Tuesday, May 5, 2015 at 8:56 AM To: CELS Admins <sysadmins(a)lists.cels.anl.gov<mailto:[email protected]>> Subject: [CELS Sysadmins] Invalid nagios config somewhere? I'm seeing lots of these messages on the hubs: Tue May 5 08:46:37 2015 : Auth: (21927) Invalid user: [nagios-radius-test-user] (from client Desktop-Server_Logins port 0) Anyone know what might be trying to use a username of 'nagios-radius-test-user' to test the hubs?

1 0

[CELS Sysadmins] Invalid nagios config somewhere?
by Leggett, Torrance I. 05 May '15

05 May '15

I'm seeing lots of these messages on the hubs: Tue May 5 08:46:37 2015 : Auth: (21927) Invalid user: [nagios-radius-test-user] (from client Desktop-Server_Logins port 0) Anyone know what might be trying to use a username of 'nagios-radius-test-user' to test the hubs?

1 0

Where's Martino
by Henderson, Martino S. 05 May '15

05 May '15

I'm running a little late and will be in around 9. Sent from My Android mobile device — Martino Henderson Help Desk / System Support Office: +1(630) 252-1692 Argonne National Laboratory Computer, Environment, & Life Sciences 9700 South Cass Avenue Building 240 - 2.E.11 Argonne, Illinois 60439 United States

1 0

Where's Tina
by Stanton-Piersanti, Tina Joy 05 May '15

05 May '15

I am working from home today. I am not feeling so well today. T

1 0

Re: [CELS Sysadmins] Services we want to retire
by Leggett, Torrance I. 04 May '15

04 May '15

Should we push to shutdown mediawiki and drupal services? From: <Stacey>, "Craig J." <stace(a)anl.gov<mailto:[email protected]>> Date: Sunday, May 3, 2015 at 3:56 PM To: CELS Admins <sysadmins(a)lists.cels.anl.gov<mailto:[email protected]>> Subject: [CELS Sysadmins] Services we want to retire One of my projects this week is writing the proposal for what our group looks like next year, two years out, five years out, etc. What services we're providing, who the primary consumer of these services are (devs, administrative users, other scientists, external collaborators? Which divisions use which share?), what services we can introduce, what extra support we can provide, etc. As part of this, I'm identifying what services we should move to CIS (some of this driven by pressure from above), what services we can look at moving to external hosting providers (RT, for instance, should ALCF want to stick with it), and what we'll continue to provide internally because it's simply the best option. While I enthusiastically welcome input on all of this, there's one area where I'm actively seeking input: what services do we want to simply retire? The one that comes immediately to mind is Bugzilla. What else is there? And, for anything you suggest, please give me a skeleton of an idea as to how we retire it? For instance, is it something we retire because there's just not enough usage to justify the effort? Is it something for which there's a better service we already provide that handles it (Bugzilla falls in this camp.)? What do you think the transition looks like? Let me know, thanks! -- Craig

2 1

[CELS Sysadmins] Services we want to retire
by Stacey, Craig J. 03 May '15

03 May '15

One of my projects this week is writing the proposal for what our group looks like next year, two years out, five years out, etc. What services we're providing, who the primary consumer of these services are (devs, administrative users, other scientists, external collaborators? Which divisions use which share?), what services we can introduce, what extra support we can provide, etc. As part of this, I'm identifying what services we should move to CIS (some of this driven by pressure from above), what services we can look at moving to external hosting providers (RT, for instance, should ALCF want to stick with it), and what we'll continue to provide internally because it's simply the best option. While I enthusiastically welcome input on all of this, there's one area where I'm actively seeking input: what services do we want to simply retire? The one that comes immediately to mind is Bugzilla. What else is there? And, for anything you suggest, please give me a skeleton of an idea as to how we retire it? For instance, is it something we retire because there's just not enough usage to justify the effort? Is it something for which there's a better service we already provide that handles it (Bugzilla falls in this camp.)? What do you think the transition looks like? Let me know, thanks! -- Craig

1 0

[CELS Sysadmins] Userbase Updates
by Leggett, Torrance I. 01 May '15

01 May '15

I just pushed quite a lot of updates to the UB2 staging server and would like help testing that things actually work as expected. The admin interface is at https://sovereign-stage.cels.anl.gov and the domain front ends are: https://accounts-stage.ci.anl.gov https://accounts-stage.jlse.anl.gov https://accounts-stage.lcrc.anl.gov https://accounts-stage.mcs.anl.gov Dan, if you want a KBase staging environment, I'll need accounts-stage.kbase.use as a CNAME for userbase-stage.cels.anl.gov.

4 11