July 2013 - Core-sysadmins - mailman3.cels.anl.gov

Resumes for Help Desk people
by Craig Stacey 13 Aug '13

13 Aug '13

Please go over the resumes in the Sharepoint Library "HelpDesk Resumes" and make note of any you think are worthwhile of a phone or in-person interview. I'll go over the resumes and your comments when I return from vacation. https://sharepoint.anl.gov/Divisions/mcs/operations/HelpDesk%20Resumes/ (I'll add a note uploading them was quite a pain since for some reason "Upload Multiple Documents" was greyed out in the "Library Tools/Documents" menu. Don't know why.) -- Craig

3 5

Where's Ti Monday
by Ti Leggett 31 Jul '13

31 Jul '13

This coming Monday we (finally!) have our pre-construction meeting for our house at 7:30 am. I'll work from home from 6:30 until I leave for the meeting and then head in afterwards and work late to make up for the time. I expect the meeting to last probably about an hour, but I'll be unreachable during that time.

1 0

IHW cleanups
by Ti Leggett 31 Jul '13

31 Jul '13

I have a few hosts that are conflicting in hostbase: This appears to be Cheryl's MBP, but that CSI is also listed in Zimbra as a loaner. Which one is correct? 140.221.11.119 (zidel-mbp.mcs.anl.gov) already associated with csi0338181.mcs.anl.gov 10:dd:b1:bc:53:10 (zidel-mbp.mcs.anl.gov) already associated with csi0338181.mcs.anl.gov erikbox was last updated on 2013-07-29 and is listed as 'Summer Student Laptop 2013: Erik Keever' and Erik Keever is active. raythunder was last updated on 2013-06-12 and has Ray Bair as the owner. Which one is correct? 140.221.10.235 (raythunder.mcs.anl.gov) already associated with erikbox.mcs.anl.gov ec:1a:59:3d:13:3f (raythunder.mcs.anl.gov) already associated with erikbox.mcs.anl.gov Hunter, can you confirm waterloo's MAC? 00:30:48:7c:9c:b6 (waterloo.mcs.anl.gov) already associated with mgt3.alcf.anl.gov

2 1

Fwd: ** RECOVERY Service Alert: pokey/www.ci.uchicago.edu is OK **
by Dan Olson 30 Jul '13

30 Jul '13

apache had gone out to lunch, probably from the network issues. I restarted it. ----- Forwarded Message ----- From: nagios(a)mcs.anl.gov To: dolson(a)mcs.anl.gov Sent: Tuesday, July 30, 2013 8:58:33 PM Subject: ** RECOVERY Service Alert: pokey/www.ci.uchicago.edu is OK ** ***** Nagios ***** Notification Type: RECOVERY Service: www.ci.uchicago.edu Host: pokey Address: pokey.ci.uchicago.edu State: OK Date/Time: Tue Jul 30 20:58:33 CDT 2013 Additional Info: HTTP OK: HTTP/1.1 200 OK - 15361 bytes in 0.067 second response time URL: https://peer.mcs.anl.gov/nagios3/cgi-bin/extinfo.cgi?type=2&host=pokey.ci.u…

1 0

Course change on accounts
by Craig Stacey 30 Jul '13

30 Jul '13

We're still proceeding on our accounts project, in that we need to true up the names and make as many accounts match as we can. But we're probably going to keep the MCS account around for the Unix infrastructure. I need to think about this while I'm away on vacation, but the meeting I had this afternoon with Pete, Rob, Marc and Paul raised dine concerns and had us doing some brainstorming. So while I'm gone, envision a system where operational systems (admin Macs, for instance) use anl accounts, and where the existing Unix infrastructure uses MCS accounts. And is also on a network generally untrusted by the rest of the lab. Think about how that might look, what stumbling blocks we'd have, etc. I'm at the CI tomorrow, but will be at Argonne early in the morning and after 5 briefly. Then I'm gone until the twelfth. -- Craig (from my mobile)

1 0

Fwd: The workspace db is growing extremely rapidly
by Dan Olson 30 Jul '13

30 Jul '13

Those of you with mongo slaves -- What is the status of the replication on mongodb? What is the available free space on the slave? ----- Forwarded Message ----- From: "Dan Olson" <dolson(a)mcs.anl.gov> To: "Christopher Henry" <chrisshenry(a)gmail.com> Cc: "Gavin A Price" <gavinaprice(a)gmail.com>, "Tom Brettin" <brettint(a)gmail.com>, "Paramvir Dehal" <psdehal(a)lbl.gov>, "Shane Canon" <scanon(a)lbl.gov>, "Stephen Chan" <sychan(a)lbl.gov>, "Neal Conrad" <nconrad(a)mcs.anl.gov> Sent: Tuesday, July 30, 2013 3:22:19 PM Subject: Re: The workspace db is growing extremely rapidly I just extended the mongodb filesystem from .5T to almost 1T. ----- Original Message ----- > From: "Christopher Henry" <chrisshenry(a)gmail.com> > To: "Dan Olson" <dolson(a)mcs.anl.gov> > Cc: "Gavin A Price" <gavinaprice(a)gmail.com>, "Tom Brettin" <brettint(a)gmail.com>, "Paramvir Dehal" <psdehal(a)lbl.gov>, > "Shane Canon" <scanon(a)lbl.gov>, "Stephen Chan" <sychan(a)lbl.gov>, "Neal Conrad" <nconrad(a)mcs.anl.gov> > Sent: Tuesday, July 30, 2013 2:58:58 PM > Subject: Re: The workspace db is growing extremely rapidly > > Well, there's a reason for this. > > 1.) I ported ALL of the model seed database to the production > workspace. That's over 19,000 models and 3000 genomes (probably > 50-60GB). > 2.) Janaka loaded all PubSEED genomes. That's over 6000 genomes, and > it accounts for 118 GB by itself. Then Janaka built core models for > all of those genomes (6,000 models) and ran FBA on 8 conditions > (8*6000 = 48,000 FBA). > > These are one-time events. They account for the bulk of the space. > And I guarantee, no one else has this much data. > > That said, this is the reason I have pushed hard to use shock to > store the full objects, as shock is supposed to find these space > demands laughable. > > The good news is, Janaka and I are stress testing the heck out of the > API, and it's holding up well. I've got a workspace with 19000 > models in it, and the workspace API and workspace browser are > handling it without a problem. Workspace browser takes maybe 10-15 > seconds to load… I consider that to be very reasonable when I've got > 19,000 models. > > We are crushing the workspace, and for good reason. If it can't hold > up to us, then it can't hold up long term to the whole community. > > There is one more big event brewing. I need to load all KBase CS > genomes and build models for them. That's going to be another 6000 > genomes and another 118 GB. That will probably happen in mid august. > I've got some cleaning up of the modeling tools before we're ready > for that. > > Chris > > P.S. This is one reason why I'm forbidding you to start from scratch > on the new workspace… because we went through alot of trouble to > load this data in. :) > P.S.S. Compression of the objects is always a possibility… but it > will hurt performance and there's no reason to do it now > > On Jul 30, 2013, at 2:42 PM, Dan Olson <dolson(a)mcs.anl.gov> wrote: > > > We have at least 10TB that we can allocate to it easily. > > > > > > ----- Original Message ----- > >> From: "Gavin A Price" <gavinaprice(a)gmail.com> > >> To: "Tom Brettin" <brettint(a)gmail.com>, "Dan Olson" > >> <dolson(a)mcs.anl.gov>, "Christopher Henry" > >> <chrisshenry(a)gmail.com>, "Paramvir Dehal" <psdehal(a)lbl.gov>, > >> "Shane Canon" <scanon(a)lbl.gov>, "Stephen Chan" > >> <sychan(a)lbl.gov> > >> Sent: Tuesday, July 30, 2013 2:34:55 PM > >> Subject: The workspace db is growing extremely rapidly > >> > >> ~ 6/20: ~50GB (deleted this dump) > >> 7/03: 91GB > >> now: 237GB > >> > >> Are we going to be able to keep up with this? > >> > >> On another note doing the switchover to the phase 1 workspace is > >> going > >> to take a ton of time just to backup the db. > >> > >> -g > >> > >> > >> > >

1 1

Username change next phase
by Craig Stacey 30 Jul '13

30 Jul '13

So Chris's form seems to do the job, so I think you guys can fire away at the list of people who need their MCS usernames changed once we get through the cleanup on the first phase. It doesn't need to wait on my return. -- Craig

2 1

Fwd: BIND DoS in the wild
by Ti Leggett 30 Jul '13

30 Jul '13

I'm going to be pushing out updates to sirius and antares this morning. Begin forwarded message: > From: Gene Rackow <rackow(a)anl.gov> > Subject: BIND DoS in the wild > Date: July 29, 2013 9:27:55 PM CDT > To: leggett(a)mcs.anl.gov, leibfritz(a)anl.gov, kvs(a)aps.anl.gov > > In case you haven't heard, on Friday ISC released a update to DNS Bind. > Turns out a simple specially crafted packet is enough to crash the server. > Patches are available for most OS's at this point in time. You should get > any server that has conduits to the outside world updated as soon as possible. > > A fwe links to find some details are: > > CVE-2013-4854 ISC FAQ and Supplemental Information > https://kb.isc.org/article/AA-01016 > > Debian Security Tracker - CVE-2013-4854 > https://security-tracker.debian.org/tracker/CVE-2013-4854 > > Red Hat Bugzilla Bug 988999 - (CVE-2013-4854) CVE-2013-4854 named will cras= > h with an assertion failure on parsing malformed rdata > https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2013-4854 > > --Gene

1 1

DRAFT: Webserver maintenance
by Dan Olson 29 Jul '13

29 Jul '13

Due to a hardware problem, we need to take one of our webservers down for maintenance this evening at 5pm central. We expect this to take less than one hour to complete, so things should be back to normal by 6pm central. The following sites / domains are affected: activecollab.cels.anl.gov events.cels.anl.gov outreach.kbase.us sc11.anl.gov wiki.globus.org www.bgconsortium.org bionet.mcs.anl.gov extremecomputingtraining.anl.gov papka.alcf.anl.gov sc12.anl.gov wiki-internal.alcf.anl.gov www.iwire.org blog.cels.anl.gov inside.cels.anl.gov press3.mcs.anl.gov scienceclouds.org wiki.iwire.org www.mpich.org bugzilla.mcs.anl.gov magellan.alcf.anl.gov press3-stage.mcs.anl.gov symposium30.cels.anl.gov wiki.kbt.mcs.anl.gov www.theseed.org cca11.org moab.mcs.anl.gov press.igsb.anl.gov tcs.anl.gov wiki.lcrc.anl.gov cpf.anl.gov nek5000.mcs.anl.gov press.mcs.anl.gov tcs.anl.gov-new wiki.magellancloud.org dev.globus.org nekcem.mcs.anl.gov pwp.alcf.anl.gov wiki.alcf.anl.gov wiki.mcs.anl.gov esp.alcf.anl.gov neklbm.mcs.anl.gov sc09.anl.gov wiki.cels.anl.gov wiki.mpich.org estrfi.cels.anl.gov outage.mcs.anl.gov sc10.anl.gov wiki.cogkit.org workshops.alcf.anl.gov ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055

1 0

Fwd: YubiRADIUS Virtual Appliance - service failed alert.
by Dan Olson 29 Jul '13

29 Jul '13

These messages were what I was referring to in the sync. ----- Forwarded Message ----- From: noreply(a)mcs.anl.gov To: root(a)mcs.anl.gov Sent: Monday, July 29, 2013 10:30:27 AM Subject: YubiRADIUS Virtual Appliance - service failed alert. YubiRADIUS Virtual Appliance version 3.5.4 "Yubico OTP Validation" service failed! Timestamp: 07/29/13 15:30:26

2 1