June 2010 - Core-students - mailman3.cels.anl.gov

[CELS Sysadmins] Fwd: SSH for limited set of users.
by stace＠mcs.anl.gov 22 Jun '10

22 Jun '10

_______________________________________________ This is the CELS Sysadmins mailing list. It is sent to memebers of: core-admins(a)mcs.anl.gov, hpc-admins(a)mcs.anl.gov, systems(a)alcf.anl.gov, aig(a)alcf.anl.gov. Do not subscribe directly to this list. Thanks.

1 0

SSH for limited set of users.
by rackow＠anl.gov 22 Jun '10

22 Jun '10

With all the SSH brute force attacks that have been happening in the last week+, a number of sites have been posting some interesting ideas on various mailing lists. One of those ideas is to run SSH on something other than port 22. The big advantage here is that on many of these attacks, they would never hit the server. Seems like a win. Unfortunately.. This would not really work for something like the general login machines as too many people would need to be trained on how to get to these servers. There is claim that it has merit for systems like pamby or other "admin-only" machines. The user community is quite small and highly technical. I'm not concerned about the issue of failed logins. For machines like these, there is such a small user community this doesn't matter. I almost think having these in the mix is just slowing them down. You are not using passwords, but OTP, so... On the other hand there have been a few cases where there has been DoS attacks that could be launched, or possibly an exploit on some really old versions of ssh. Running on an odd port would keep most of the scanners searching for this kind of hole from finding the machine at all. Anyway, this is something worth considering. On one hand it's security by obscurity which never really works. On the other it's removing the bulk of the attack noise you know is just going to fail anyway. --Gene

1 0

Re: interest in GPFS site license?
by dolson＠mcs.anl.gov 22 Jun '10

22 Jun '10

I briefly thought about how to best make our nfs environment HA without a SAN - without getting away from ZFS. As far as I can tell there is no way to do that with a filesystem based clustered filesystem. Basically we need a clustered block device. ZFS will work with filesystem backed vdevs but everything I've read says that is only for testing. There is a possible (maybe crazy) way to do this with iscsi - for ex on the thumpers On thumper 1 half the disks are set up in a zpool (zpool1) On thumper 2 half the disks are set up in zpool2 Share the other half out iscsi on both with the 10gbe interconnect between the two. Zfs mirror each local disk half with the iscsi devs presented from the other thumper On failover import the failed zpool and ifalias its ip on the local nic. --- GPFS also is only supported on linux with redhat or suse. We could load up some front end boxes with redhat/suse and iscsi mount the thumpers. It would introduce some complexity. When the drives come in for quicksilver I'm going to be doing a bunch of testing, so I wouldn't mind giving it a try and seeing what it can do. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Ti Leggett" <leggett(a)mcs.anl.gov> To: stace(a)mcs.anl.gov Cc: "Core Admins" <core-admins(a)mcs.anl.gov> Sent: Tuesday, June 22, 2010 8:33:49 PM GMT -06:00 US/Canada Central Subject: Re: interest in GPFS site license? GPFS is most useful with FC or IB (maybe iSCSI) disk and if used that way it's very resilient and very available. Downtimes can be greatly minimized and loads can be more evenly distributed. I'd be interested to see if a clustered NFS backed GPFS filesystem might help some of the load issues Dan's been fighting. On Jun 22, 2010, at 8:03 PM, stace(a)mcs.anl.gov wrote: > Now this is interesting. My gut is that we are, but are we really? Opinions? > <Mail Attachment.eml> >

1 0

Output from "cron" command
by Super-User 22 Jun '10

22 Jun '10

Your "cron" job on mercury /root/bin/daily-snapshot.sh produced the following output: cannot open 'zpool1/sandboxes@Tuesday': dataset does not exist

1 0

Where's Tina (Thursday, 6/22/2010)
by Tina J. Stanton-Piersanti 22 Jun '10

22 Jun '10

This Thursday I will be leaving at lunch and going to sign up for Vinnie for school and then to our refinancing closing. I finish out the rest of hours from home that afternoon. T. Begin forwarded message: > From: stace(a)mcs.anl.gov > Date: June 22, 2010 2:43:43 PM CDT > To: "Tina J. Stanton-Piersanti" <tstanton(a)mcs.anl.gov> > Subject: Re: Heads Up.. > > Yup > > "Tina J. Stanton-Piersanti" <tstanton(a)mcs.anl.gov> wrote: > > We are closing at 2:00 on Thursday.. so I am going to leave at lunch, > go try to get Vinnie signed up for school, go to the closing and > finish out the rest of my hours at home.. of course with your approval > first. > > Just let me know > > > T. > > On Jun 21, 2010, at 9:25 AM, Tina J. Stanton-Piersanti wrote: > >> We are refinancing and if we close this week before Thursday we can >> get a rate of 4.62. Going to need a couple hours off to do this.. So >> the bank is supposed to call Joe with a date and time. Just awaiting >> on confirmation for that. If it early enough in the day on the day >> they can do it, I still need to register Vinnie for the public >> school. So I will keep you posted, >> >> T. >

1 0

Maintenance day schedule
by Ti Leggett 22 Jun '10

22 Jun '10

Here's what I was thinking of sending to the CI about maintenance schedule. Please feedback so I can send this out ASAP: In an effort to maintain security, we have developed a maintenance schedule to keep all CI machines up to date with vendor patches. In order to reduce the impact of these maintenance windows, we have split them up into four categories of machines to be handled on different days once a month: HPC Machines (Teraport & PADS): First Tuesday of every month starting at 7:00am Desktop Machines: Second Tuesday of every month starting at noon but delaying needed reboots until after 5:00pm Login server & Project servers (communicado, sidgrid, BIRN, CABig, etc): Third Tuesday of every month starting at 7:00am Core servers (mail, web, file, VM, PADS GPFS, etc): Third Saturday of every month starting at 7:00 am. We will send email one week prior to any maintenance if we are going to perform the maintenance. If the maintenance window will severely impact time critical work that month, please let us know ASAP so we can reschedule or expedite certain services or machines. We plan to start implementing the in July, starting on July 6th. While we understand how disruptive these maintenances can be, a security compromise is far more disruptive. But, as always, we welcome any feedback on this schedule.

3 2

Re: [backup-admins] MySQL@spock backup
by dolson＠mcs.anl.gov 22 Jun '10

22 Jun '10

Narayan, It looks like were stuck on the version we are running. I'm left with truncating that database by id or dropping it. Would dropping that database affect bcfg2's functionality outside of reporting? ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Original Message ----- From: "Narayan Desai" <desai(a)mcs.anl.gov> To: "Dan Olson" <dolson(a)mcs.anl.gov> Cc: core-admins(a)mcs.anl.gov Sent: Tuesday, June 22, 2010 8:38:06 AM GMT -06:00 US/Canada Central Subject: Re: [backup-admins] MySQL@spock backup You can run this with "bcfg2-admin reports scrub", but it only showed up after rc2, so you'll need to grab the latest tarball from ftp://ftp.mcs.anl.gov/pub/bcfg/bcfg2-1.1.0rc3.tar.gz -nld On Jun 22, 2010, at 8:24 AM, Dan Olson wrote: > Ok, I'll give that a shot first. > > ---- > Daniel Murphy-Olson > Systems Administrator > Mathematics & Computer Science Division > Argonne National Laboratory > 630-252-0055 > > ----- Original Message ----- > From: "Narayan Desai" <desai(a)mcs.anl.gov> > To: "Dan Olson" <dolson(a)mcs.anl.gov> > Cc: core-admins(a)mcs.anl.gov > Sent: Tuesday, June 22, 2010 8:10:21 AM GMT -06:00 US/Canada Central > Subject: Re: [backup-admins] MySQL@spock backup > > there is a new compressor/deduplicator command in bcfg2-1.1.0rc3 that you might try running on the db; it should really cut down on the amount of data that is stored. > -nld > > On Jun 22, 2010, at 8:07 AM, Dan Olson wrote: > >> Is anyone using the data in the bcfg2_reports database on spock? If I don't get any replies I'm going to archive it and clear it out. >> >> ---- >> Daniel Murphy-Olson >> Systems Administrator >> Mathematics & Computer Science Division >> Argonne National Laboratory >> 630-252-0055 >> >> ----- Forwarded Message ----- >> From: "root" <root(a)mcs.anl.gov> >> To: backup-admin(a)mcs.anl.gov >> Sent: Monday, June 21, 2010 10:55:48 PM GMT -06:00 US/Canada Central >> Subject: [backup-admins] MySQL@spock backup >> >> mysqldump: Got errno 28 on write >> >> gzip: /sandbox/backup/mysql_dbs/bcfg2_reports.20100621220002.dump.gz: No space left on device >> mysqldump: Got errno 28 on write >> >> gzip: /sandbox/backup/mysql_dbs/mysql.20100621220002.dump.gz: No space left on device >> Backup # 1 = bcfg2_reports.20100621220002.dump -> 46809354240 bytes >> Backup # 2 = bcfg2_reports.20100620220001.dump -> deleted >> Backup # 1 = information_schema.20100621220002.dump.gz -> 465 bytes >> Backup # 2 = information_schema.20100620220001.dump.gz -> deleted >> Backup # 1 = mysql.20100621220002.dump -> 0 bytes >> Backup # 2 = mysql.20100620220001.dump -> deleted >> >> >

2 1

Fwd: [backup-admins] MySQL@spock backup
by Dan Olson 22 Jun '10

22 Jun '10

Is anyone using the data in the bcfg2_reports database on spock? If I don't get any replies I'm going to archive it and clear it out. ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055 ----- Forwarded Message ----- From: "root" <root(a)mcs.anl.gov> To: backup-admin(a)mcs.anl.gov Sent: Monday, June 21, 2010 10:55:48 PM GMT -06:00 US/Canada Central Subject: [backup-admins] MySQL@spock backup mysqldump: Got errno 28 on write gzip: /sandbox/backup/mysql_dbs/bcfg2_reports.20100621220002.dump.gz: No space left on device mysqldump: Got errno 28 on write gzip: /sandbox/backup/mysql_dbs/mysql.20100621220002.dump.gz: No space left on device Backup # 1 = bcfg2_reports.20100621220002.dump -> 46809354240 bytes Backup # 2 = bcfg2_reports.20100620220001.dump -> deleted Backup # 1 = information_schema.20100621220002.dump.gz -> 465 bytes Backup # 2 = information_schema.20100620220001.dump.gz -> deleted Backup # 1 = mysql.20100621220002.dump -> 0 bytes Backup # 2 = mysql.20100620220001.dump -> deleted

2 3

Where's Rick
by bradshaw＠mcs.anl.gov 22 Jun '10

22 Jun '10

I woke up with swollen glands and a sore throat. I am going to work from home but if things get worse I will take a sick day. I will be available via mudd, Im, email or cell Rick

1 0

Output from "cron" command
by Super-User 21 Jun '10

21 Jun '10

Your "cron" job on mercury /root/bin/daily-snapshot.sh produced the following output: cannot open 'zpool1/sandboxes@Monday': dataset does not exist

1 0