August 2009 - Core-sysadmins - mailman3.cels.anl.gov

Hunter for Aug 29th
by Hunter Matthews 31 Aug '09

31 Aug '09

Monday Meeting on IGSB dna storage issues. Asked Jarred to write a quick storage map for existing storage Started "draining" DMOJ (damage) to fs02.cosmea after the lustre cluster move did about .5TB over the weekend (far too slow). Tuesday Kept working on dmoj as it has a tendancy to crap out rsync jobs With jarred's help have all 5 racks preped for the igsb/nmpdr moves Wednesday Started move of IGSB equipment to new racks Trouble with the ACT rails - they're too long Started drilling extra holes in the rails Thursday Stopped drilling extra holes - the servers didn't fit with the extra holes Racked these servers in a borrowed APC brand (square hole) rack. Started trying to get Lustre cluster back up. Friday Finally got lustre cluster back up. Helped a little with server room cleanup for open house Saturday Open house for 2 hours. -- Hunter Matthews Unix Administrator Office: Bldg 221 Room B240 Argonne National Labs, MCS Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438 Never take candy from strangers. Especially on the internet.

1 0

update for ken 8/31
by Kenneth Raffenetti 31 Aug '09

31 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - - lots of email2trac debugging for mpich, this randomly stopped working, workaround in place but we discovered a bug in the ticket notification today - - worked with Duane regarding new displays in the front of the SC booth - - finished setting up lists.lcrc.anl.gov Mailman vhost - - hacked off broken bolt on 65" plasma in hopes of getting it fixed soon - - tossed/excessed a bunch of stuff from helpdesk cabinets - - random reqs - - advanced to finals of Reactor Golf League team playoffs :) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkqcSlQACgkQvRJB9CBtvwgPegCgu9cP07a5+AQSKlK4IHBEaWLz g6UAn2+/tg9sgW7U0De2QzPqT14rs6SX =Vlr7 -----END PGP SIGNATURE-----

1 0

Phone Death
by Hunter Matthews 24 Aug '09

24 Aug '09

To cap off the end of the perfect day, my nearly new iphone ended up taking a bath in a cherry flavored drink cup on the way home. Its currently "drying" in the kitchen, where "drying" is probably dying as it turns itself on an off every 2-3 minutes. I can't get the cherry drink out - I've pulled the SIM card and have it upside down to drain, but thats all I can do. If you need to call me, you'll have to use my office or google number (I don't have a house phone). -- Hunter Matthews hunter.matthews(a)gmail.com Unix Administrator, Photographer Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438 Never take candy from strangers. Especially on the internet.

1 0

update for ken 8/24
by Kenneth Raffenetti 24 Aug '09

24 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - - i2u2 re-racking - - upgraded data1.i2u2.org from Sarge to Hardy - - meeting w/ Duane and Joe about SC09 booth equipment - - started to track down a Panasonic repair shop to service SC plasma screen - - IGSB re-rack planning meeting - - put up emergency Computer Science Review website - - helpdesk week req foo Ken -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkqTDPYACgkQvRJB9CBtvwhvQwCgnVtCGlYJvpYqT02xwrwpG31n IVIAoKPkc+ustxKhHx2H+Tz/5aAto5YH =hn58 -----END PGP SIGNATURE-----

1 0

Matthews report for Aug 21
by Hunter Matthews 24 Aug '09

24 Aug '09

Monday IGSB re-racking meeting Started sorting igsb/nmpdr spares from bills cube Installed and starting configuring new install/infrastructure server for igsb/nmpdr (warehouse13.mcs.anl.gov) IGSB user issue. Tuesday XM travel meeting (out early) Configuring warehouse Cleaning igsb/nmpdr racks NMPDR re-racking meeting Still fixing dvir disk space issues Two small Cosmea issues. Wednesday Unracking TG racks to prep for re-racking IGSB Meeting on IGSB storage Configuring warehouse Finished with Bill's Cube, started on A005 DVIR's storage problem was that the flexstorage head node was off, but I didn't realize it. Once that was rebooted, iscsi came up and gpfs remounted the filesystem without issue. Thursday Unracking TG racks to prep for re-racking NMPDR (more to do) Finished with warehouse and mrepo, Finished with A005 Friday Meeting/discussion on IGSB dna storge -- Hunter Matthews Unix Administrator Office: Bldg 221 Room B240 Argonne National Labs, MCS Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438 Never take candy from strangers. Especially on the internet.

1 0

Fwd: [CI Ticketing System #1556] Need a tiny virtual machine lab for testing PBS workflows
by Ti Leggett 21 Aug '09

21 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Here's the explanation for why it got hacked. Begin forwarded message: > From: "labisso(a)uchicago.edu" <support(a)ci.uchicago.edu> > Date: August 21, 2009 2:00:27 PM CDT > To: undisclosed-recipients:; > Subject: Re: [CI Ticketing System #1556] Need a tiny virtual machine > lab for testing PBS workflows > Reply-To: support(a)ci.uchicago.edu > > > Hi all- > > We're pretty sure we have found the source of the exploit and I am > ashamed > to admit that it is entirely my fault. I made a VM image for Bill > Mihalo a > few days ago and I forgot to turn off password-based logins on it. An > account had a weak default password that I also neglected to change. > It > seems very likely that the VM was compromised by a brute force SSH > attack. > These are extremely common. > > Unfortunately there are not logs available on the image for this time > period. Although it appears that the image was saved when it was > shut down, > there is a suspicious gap in the logs. It is quite possible that > records > were deleted. > > In my opinion, this attack was isolated and not a result of a > vulnerability > in the Nimbus service or service nodes. > > I removed the compromised image from the repository, but saved a copy > elsewhere for forensics. Clearly it cannot be used anymore as some > backdoors > could have been created. Additionally, I will audit all other images > in our > repository to ensure that password-logins are disabled. > > Ti, if you are satisfied, I would like to bring the Nimbus service > back > online. > > I am extremely sorry for this mistake and any problems it has caused. > > David > > On Fri, Aug 21, 2009 at 12:32 PM, David LaBissoniere > <labisso(a)uchicago.edu>wrote: > >> We're pretty sure we found the cause of the exploit, and luckily it >> appears >> to have been limited to that VM. I'll send a full explanation >> shortly. >> >> David >> >> >> On Fri, Aug 21, 2009 at 11:39 AM, David LaBissoniere <labisso(a)uchicago.edu >>> wrote: >> >>> The Nimbus service is down and all VMs are terminated. The service >>> nodes >>> are still running however. >>> >>> This means, all tp-x** nodes are down, but the tp-c** nodes are >>> still >>> running. >>> >>> I'm attempting to correlate the UT report with our logs. >>> >>> David >>> >>> >>> On Fri, Aug 21, 2009 at 11:30 AM, Ti Leggett <support(a)ci.uchicago.edu >>> >wrote: >>> >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> Shut down the service as in the network is down completely? If not, >>>> that's what must happen. >>>> >>>> We received a report from the University of Texas. Here are the >>>> details they sent over. >>>> >>>> tp-x003.ci.uchicago.edu ( 128.135.125.23 ) scanned our >>>> 128.125.0.0/16 >>>> network, >>>> accessing ssh ports from 18:25 to 18:26 PDT/PST (GMT - 0700/0800) >>>> Over 27000 hosts were probed. This indicates that you have a >>>> hacker on >>>> the system who is trying to compromise computer security, and who >>>> may >>>> have compromised your host. Your host may be blocked pending your >>>> reply. >>>> >>>> On Aug 21, 2009, at 11:24 AM, Kate Keahey wrote: >>>> >>>>> >>>>> Ti, >>>>> >>>>> David shut down the service and we are looking. What are the exact >>>>> times >>>>> of the login attempts? >>>>> >>>>> David LaBissoniere wrote: >>>>>> Pulling network on tp-x003 is fine. But the physical machine is >>>>>> called >>>>>> tp-c003. I will remove it from the Nimbus pool in the meantime. >>>>>> >>>>>> Unfortunately it has already had some other VMs run on that >>>>>> service >>>>>> node >>>>>> this morning. (Some of mine, at least.) >>>>>> >>>>>> [I trimmed Mike Wilde and Yi from the CC list, and added Kate >>>>>> Keahey and >>>>>> Tim Freeman] >>>>>> >>>>>> David >>>>>> >>>>>> On Fri, Aug 21, 2009 at 10:51 AM, Ti Leggett <support(a)ci.uchicago.edu >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> To be clear tp-x003 should have its network pulled and left >>>>>> to be >>>>>> forensically combed over. All other nodes should look for >>>>>> signs of >>>>>> compromise and then upgraded and rebooted ASAP. If I don't here >>>>>> in an >>>>>> hour, I'm going to pull the network on all 16 machines. >>>>>> >>>>>> On Aug 21, 2009, at 10:49 AM, Mike Wilde wrote: >>>>>> >>>>>>> >>>>>>> David, Bill, >>>>>>> >>>>>>> Can you confirm to Ti that you or someone from Nimbus is >>>>>>> acting on >>>>>>> Ti's >>>>>>> request to patch VMs? >>>>>>> >>>>>>> I'm going to step out of the discussion, as I'm only involved in >>>>>>> that I >>>>>>> *want* a new VM. >>>>>>> >>>>>>> David, can you let me know when it is safe to actually proceed >>>>>> with my >>>>>>> VM setup as you suggested earlier? >>>>>>> >>>>>>> Lastly - Yi, do you need to apply security patches to any of >>>>>>> the VMs >>>>>>> that you run in EC2? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Mike >>>>>>> >>>>>>> >>>>>>> On 8/21/09 10:25 AM, Ti Leggett wrote: >>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>>> Hash: SHA1 >>>>>>>> >>>>>>>> Bill Mihalo was on the machine last night about 20 minutes >>>>>> before the >>>>>>>> apparent attack happened. I've included him to provide more >>>>>>>> information. >>>>>>>> >>>>>>>> Bottom line is this: Stop and update every host and VM that you >>>>>>>> have >>>>>>>> to the latest security patches right now. Included in that is >>>>>>>> especially the latest kernel. You can either do this now, or >>>>>> shut the >>>>>>>> machines down. Those are the options. >>>>>>>> >>>>>>>> On Aug 21, 2009, at 9:12 AM, labisso(a)uchicago.edu >>>>>> <mailto:[email protected]> wrote: >>>>>>>> >>>>>>>>> I don't see any VM running on tp-x003 and from what I can >>>>>>>>> tell, >>>>>>>>> nothing has >>>>>>>>> run on that node since about 6pm last night. What is the >>>>>>>>> timeframe >>>>>>>>> for the >>>>>>>>> login attempts? I can drill into the logs further and see >>>>>>>>> who/what >>>>>>>>> was >>>>>>>>> running at that time. >>>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>> On Fri, Aug 21, 2009 at 8:55 AM, David LaBissoniere >>>>>> <labisso(a)uchicago.edu <mailto:[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> tp-x003 is a VM running on service node running on tp-c003. >>>>>>>>>> I'm >>>>>>>>>> trying to >>>>>>>>>> see what is running and shut down the VM. Tim? >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Aug 21, 2009 at 8:30 AM, Ti Leggett >>>>>>>>>> <support(a)ci.uchicago.edu >>>>>>>>>> <mailto:[email protected]>>wrote: >>>>>>>>>> >>>>>>>>>>> Are you running by chance on tp-x003? If so you need to stop >>>>>>>>>>> immediately. I just got a call from NSIT security that that >>>>>>>>>>> machine is >>>>>>>>>>> trying repeated root logins both on campus and off campus. >>>>>>>>>>> If I >>>>>>>>>>> haven't >>>>>>>>>>> heard anything by the time I get on campus, I'm going to >>>>>> power the >>>>>>>>>>> machine off. >>>>>>>>>>> >>>>>>>>>>> On Thu Aug 20 18:52:54 2009, wilde(a)mcs.anl.gov >>>>>> <mailto:[email protected]> wrote: >>>>>>>>>>>> Ti, >>>>>>>>>>>> >>>>>>>>>>>> I just met with David, and I think I can get what I want >>>>>> from the >>>>>>>>>>>> TeraPort Nimbus cluster, so David will set that up for me >>>>>> and you >>>>>>>>>>>> can >>>>>>>>>>>> close this ticket. >>>>>>>>>>>> >>>>>>>>>>>> David, my cert CN is: >>>>>>>>>>>> /DC=org/DC=doegrids/OU=People/CN=Michael Wilde 681533 >>>>>>>>>>>> >>>>>>>>>>>> Yi, do you have a virtual cluster that includes PBS on >>>>>>>>>>>> which I >>>>>>>>>>>> can run >>>>>>>>>>>> Swift? Can you point us to this so that I can clone it >>>>>>>>>>>> for the >>>>>>>>>>>> Nimbus >>>>>>>>>>>> cluster? Did you already run your virtual cluster or grid >>>>>>>>>>>> under >>>>>>>>>>>> Nimbus? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Mike >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 8/20/09 6:37 PM, General CI Support Tickets wrote: >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> Please read the entire message. We have a number of quick >>>>>>>>>>>>> tips at >>>>>>>>>>>> the >>>>>>>>>>>>> bottom of this message that may just answer your >>>>>>>>>>>>> question and >>>>>>>>>>>>> get >>>>>>>>>>>> you >>>>>>>>>>>>> up and running sooner. >>>>>>>>>>>>> >>>>>>>>>>>>> This message has been automatically generated in response >>>>>> to the >>>>>>>>>>>>> creation of a trouble ticket regarding: >>>>>>>>>>>>> "Need a tiny virtual machine lab for testing PBS >>>>>>>>>>>>> workflows", >>>>>>>>>>>>> a summary of which appears below. >>>>>>>>>>>>> >>>>>>>>>>>>> There is no need to reply to this message right now. Your >>>>>>>>>>>>> ticket >>>>>>>>>>>> has been >>>>>>>>>>>>> assigned an ID of [CI Ticketing System #1556]. >>>>>>>>>>>>> >>>>>>>>>>>>> Please include the string: >>>>>>>>>>>>> >>>>>>>>>>>>> [CI Ticketing System #1556] >>>>>>>>>>>>> >>>>>>>>>>>>> in the subject line of all future correspondence about >>>>>>>>>>>>> this >>>>>>>>>>>>> issue. >>>>>>>>>>>> To do so, >>>>>>>>>>>>> you may reply to this message. >>>>>>>>>>>>> >>>>>>>>>>>>> This e-mail address is for reporting problems in the CI >>>>>>>>>>>>> core >>>>>>>>>>>> computing >>>>>>>>>>>>> infrastructure. If your request is not related to that, >>>>>>>>>>>>> we'll >>>>>>>>>>>>> make >>>>>>>>>>>> sure it >>>>>>>>>>>>> reaches the right people. However, for future reference, >>>>>> please >>>>>>>>>>>> make note of >>>>>>>>>>>>> the addresses below to ensure a faster response: >>>>>>>>>>>>> >>>>>>>>>>>>> Teraport: teraport-support(a)ci.uchicago.edu >>>>>> <mailto:[email protected]> >>>>>>>>>>>>> >>>>>>>>>>>>> PADS: pads-support(a)ci.uchicago.edu >>>>>> <mailto:[email protected]> >>>>>>>>>>>>> >>>>>>>>>>>>> "My account has been disabled." >>>>>>>>>>>>> or >>>>>>>>>>>>> "I forgot my password." >>>>>>>>>>>>> >>>>>>>>>>>>> If your account has been disabled or you have forgotten >>>>>>>>>>>>> your >>>>>>>>>>>> password, >>>>>>>>>>>>> you must contact the help desk by calling 773-834-4102 >>>>>>>>>>>>> between >>>>>>>>>>>>> 8:30AM and 11:30AM or 1:00PM and 4:30PM Monday through >>>>>>>>>>>>> Friday >>>>>>>>>>>>> to have it reactivated. >>>>>>>>>>>>> >>>>>>>>>>>>> "How can I have my SSH key activated for my account?" >>>>>>>>>>>>> >>>>>>>>>>>>> http://www.ci.uchicago.edu/support/sshkeys >>>>>>>>>>>>> >>>>>>>>>>>>> "How do I forward/stop forwarding my mail?" >>>>>>>>>>>>> >>>>>>>>>>>>> http://www.ci.uchicago.edu/wiki/bin/view/Resources/ >>>>>>>>>>>>> MailForwarding >>>>>>>>>>>>> >>>>>>>>>>>>> "How do I set an out of office or vacation message?" >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>> http://www.ci.uchicago.edu/wiki/bin/view/Resources/VacationMessage >>>>>>>>>>>>> >>>>>>>>>>>>> "How can I request a database?" >>>>>>>>>>>>> >>>>>>>>>>>>> http://www.ci.uchicago.edu/support/databases >>>>>>>>>>>>> >>>>>>>>>>>>> "How can I request a mailing list?" >>>>>>>>>>>>> >>>>>>>>>>>>> http://www.ci.uchicago.edu/support/lists >>>>>>>>>>>>> >>>>>>>>>>>>> Other helpful URLS: >>>>>>>>>>>>> >>>>>>>>>>>>> http://www.ci.uchciago.edu/faq >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>> >>>> http://www.ci.uchicago.edu/wiki/bin/view/Resources/CheckingSendingMail >>>>>>>>>>>> >>>>>> http://www.ci.uchicago.edu/wiki/bin/view/Resources/CalendaringFAQ >>>>>>>>>>>>> http://www.ci.uchicago.edu/wiki/bin/view/Resources/Softenv >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you, >>>>>>>>>>>>> CI Support >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>> >>>>>> >>>> ------------------------------------------------------------------------- >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> Can I get, or clone, a small Nimbus workspace or similar >>>>>>>>>>>>> lab >>>>>>>>>>>>> that I >>>>>>>>>>>> can >>>>>>>>>>>>> run on bridled or tron, on which I can do rapid testing of >>>>>> Swift >>>>>>>>>>>>> for >>>>>>>>>>>> PBS >>>>>>>>>>>>> workflows? >>>>>>>>>>>>> >>>>>>>>>>>>> I want something where I can regulate the number of >>>>>>>>>>>>> cores PBS >>>>>>>>>>>>> thinks >>>>>>>>>>>> it >>>>>>>>>>>>> has (from say 1 to 10) and have it get up to 4 cores >>>>>>>>>>>>> active on >>>>>>>>>>>> bridled >>>>>>>>>>>>> or whatever tron has. >>>>>>>>>>>>> >>>>>>>>>>>>> Ideally this can be a single vm, or a small virtual >>>>>>>>>>>>> cluster >>>>>> (eg, >>>>>>>>>>>>> 2 to >>>>>>>>>>>> 10 >>>>>>>>>>>>> VMs). >>>>>>>>>>>>> >>>>>>>>>>>>> Does anything exist like this right now that I can >>>>>>>>>>>>> either use >>>>>>>>>>>>> as- >>>>>>>>>>>>> is, >>>>>>>>>>>> or >>>>>>>>>>>>> clone to form my own VM? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Mike >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>> -----BEGIN PGP SIGNATURE----- >>>>>>>> Version: GnuPG v1.4.8 (Darwin) >>>>>>>> >>>>>>>> iEYEARECAAYFAkqOvHAACgkQ4RgdOxQVi0AlSwCeM7lTU7rQmWPn1y/OWh >>>>>>>> +7qKs6 >>>>>>>> jvAAnRpwwTMOSQ2g62Y9r9XwvCuoqu5N >>>>>>>> =68Zt >>>>>>>> -----END PGP SIGNATURE----- >>>>>>>> >>>>>>> >>>>>> >>>>>> -----BEGIN PGP SIGNATURE----- >>>>>> Version: GnuPG v1.4.8 (Darwin) >>>>>> >>>>>> >>>>>> iEYEARECAAYFAkqOwn0ACgkQ4RgdOxQVi0DggACdF77PUEnqVVEOpbrzgbETtWbu >>>>>> DEMAnjZksHvgBIAELRl9dJNhJUZ8EU7w >>>>>> =eFJP >>>>>> -----END PGP SIGNATURE----- >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> Kate Keahey, >>>>> Mathematics & CS Division, Argonne National Laboratory >>>>> Computation Institute, University of Chicago >>>>> >>>>> >>>> >>>> -----BEGIN PGP SIGNATURE----- >>>> Version: GnuPG v1.4.8 (Darwin) >>>> >>>> iEYEARECAAYFAkqOy7wACgkQ4RgdOxQVi0ByKgCfTSomnaa3Tnt/VRjOhBz62gP6 >>>> KzQAn3TBq/6/qTNPbeYcivFJlm40Ua2r >>>> =+QuT >>>> -----END PGP SIGNATURE----- >>>> >>>> >>> >> > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkqO9UEACgkQ4RgdOxQVi0CWkwCfTsU8cM87ZoiHMBux51e2wGku Q+oAn26tU15hXhCfK2F3tc/Au02kOiqw =lgIA -----END PGP SIGNATURE-----

1 0

Fwd: [UCHI-NSC21854] - Your Network Access Has Been Disabled - some network forensic information
by Ti Leggett 21 Aug '09

21 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Here's some info NSIT was able to provide. Begin forwarded message: > Hello Ti, > > Although we cannot tell for sure, the attack appears to have been an > SSH > brute-force attack intially from a Russian IP address 213.85.77.92 > that > began around 2009-08-20 20:12hr CST. > > Within about 10 minutes, it appears that a few files were downloaded > onto 128.135.125.23: > - W2Ksp3.exe from download.microsoft.com > - gosh.tgz from mala69.ucoz.com, > - eduard.tgz from linusovy.tripod.com > > The SSH scanning begins directly after that in a sequential scan > starting with 128.0.0.1. > > Regards, > Shelley -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkqO2vMACgkQ4RgdOxQVi0DYaACfUm4IY5u/JjhGVLaTl+y2Wr2P sRIAnjzMMkLkzDTUOUN4EpvlRwm3BiT7 =SBVs -----END PGP SIGNATURE-----

1 0

Security Compromise at CI
by Ti Leggett 21 Aug '09

21 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The machine tp-x003.ci.uchicago.edu (128.135.125.23), which is a Nimbus Xen VM on tp-c003.ci.uchicago.edu was compromised. At around 6:25pm it started root scanning CI hosts and hosts at UTexas. The good news is that the CI doesn't manage tp-x003 or tp-c003 (i.e., it's not tied to the CI auth system, NFS system, etc), so this compromise doesn't mean much to us other than it was on the same network as other CI machines. It appears that host of origin may have been an NCAR host, 128.117.43.83. For the time being I would block both of those hosts. Kate's admin, David LaBissio, is working on getting all the other numbus servers (tp-x001 - x016) updated to latest patches, and I'm requesting forensics from him on the vector of attack and entry and escalation (if present). -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkqO2fAACgkQ4RgdOxQVi0BYhQCbBcafaVPg90L34N14Bu9ij/HZ 5PUAnjziaeAXjHIuSteg1TgBdFhHtPfW =7e0W -----END PGP SIGNATURE-----

1 0

Fwd: Linux NULL pointer dereference due to incorrect proto_ops initializations
by Craig Stacey 21 Aug '09

21 Aug '09

== Craig Begin forwarded message: > From: Zhen Xie <zhenxie(a)mcs.anl.gov> > Date: August 19, 2009 7:54:31 PM CDT > To: Craig Stacey <stace(a)mcs.anl.gov> > Subject: Linux NULL pointer dereference due to incorrect proto_ops > initializations > Reply-To: Zhen Xie <zhenxie(a)mcs.anl.gov> > > Hi Craig, > > http://archives.neohapsis.com/archives/fulldisclosure/ > 2009-08/0174.html > http://blog.cr0.org/2009/08/linux-null-pointer-dereference-due-to.html > > Maybe you are already aware of this, just in case. Seems a critical > security > issue for Linux system. > > Zhen

3 4

Interviewing IBI/CI Sysadmins
by Craig Stacey 19 Aug '09

19 Aug '09

On Thursday, a couple of sysadmins from the IBI group will be coming out to Argonne to interview for new positions with the CI as part of the CI absorbing IBI. At the very least, Ti and I will need to interview them, but if you're available for an interview, let me know. Tina or I will come up with an interview schedule. I'll forward on resumes once I get them, as well as pointers to the posting. == Craig

2 2