January 2015 - Core-sysadmins - mailman3.cels.anl.gov

[CELS Sysadmins] Puppet meeting followup
by Christopher P. Bills 23 Jan '15

23 Jan '15

As we discussed in the meeting, the plan going forward with Puppet is to take in the module changes I've been working on, and step back which modules are in active use, until we can more thoroughly test them. We will start by ensuring 'base' is safe on all hosts, and then applying the base module to all hosts. The next step is to 'audit' the remaining modules, and do whatever testing needs to be done on them, to ensure we can trust them to run every "30 minutes" or whatever time we deem appropriate. To get to the first part, I will make changes to the dev_cpb branch to classify hosts with hiera, and stop using nodes.pp to manage that, and do some closer looking at 'base' to see if it needs anything before we go forward with using it. Once I'm satisfied with that, I'll rebase off the mcs branch and handle any merge conflicts, and point mcs at the same commit as cpb_dev, and we will be caught up, and further dev work with Puppet configuration can mimic (or improve on) the style I've been using. To facilitate multiple environments, we discussed potentially using generic modules wherever possible, but when things are specific to an environment, naming them as such; chameleon::rsyslog, for example, and ensuring those modules are only included in the environments they belong to. This enables us to share a common base, as much as possible, while allowing environments to treat similar services slightly differently. The long-term plan for Puppet is to identify and 'mark' a selection of production systems as testing, stage, and stable (or some other terminology), where testing is the smallest group, and gets the most current changes from Puppet, and changes are promoted from that point on to stable. To help with that, it will be nice to have a custom / managed repository with similar test/stage/stable promotion of packages, as they are updated upstream. Pulp is a tool that has been used at ORNL, and there is a paper about how they were using it (link below), and it could likely work well for our needs, as well. Also, we discussed documentation, and the idea of marking out a few days a week to migrate documentation from other sources, maybe 10 documents at a time, until we finally get any needed documentation into Confluence. To handle the event when we cannot access Confluence, but need access to our documentation, we are looking into automating an export of the entire wiki to PDF and pushing it to BOX. Once all documentation has been migrated to Confluence and we have the PDF automation handled, we should purge and burn with fire the older documentation systems to ensure we use Confluence going forward, and don't end up with split-brain all over again. Sorry, that was really wordy, but I wanted to lay out the plans as I see them. I've created a handful of JIRA issues, linked below. https://xjira.mcs.anl.gov/browse/CELS-14 - Create local repo https://xjira.mcs.anl.gov/browse/CELS-13 - Make Puppet automatic-able https://xjira.mcs.anl.gov/browse/CELS-15 - Establish tiered envs https://xjira.mcs.anl.gov/browse/CELS-28 - Bring Puppet updates in https://xjira.mcs.anl.gov/browse/CELS-29 - Automate PDF of Wiki https://www.usenix.org/legacy/events/lisa11/tech/full_papers/Pierre.pdf Paper by the guy at ORNL about using Pulp for packages and staged promotion. https://collab.mcs.anl.gov/display/CELS/CELS+Systems The CELS 'Wiki'

1 0

Where's Martino
by Henderson, Martino S. 22 Jan '15

22 Jan '15

I have a few personal matters to take care of today. I may not be in at all but will try to finish as soon as possible. Sent from My Android mobile device — Martino Henderson Help Desk / System Support Office: +1(630) 252-1692 Argonne National Laboratory Computer, Environment, & Life Sciences 9700 South Cass Avenue Building 240 - 3.E.18 Argonne, Illinois 60439 United States

1 0

[CELS Sysadmins] tape library downtime?
by John Valdes 21 Jan '15

21 Jan '15

Sysadmins, LCRC recently purchased some tape drives (and tapes) for the SpectraLogic library in the DC for backing up our newer filesystems, and the plan is to create a new partition in the library to contain them. To create the new partition, however, it is best to take the library offline. LCF can do this this coming Monday (1/26) if that day works for the rest of us. Do you have any issues/conflicts with having the library offline on Monday? I don't know how long it will take to create the new partition, but I couldn't imagine it taking more than an hour or two. While offline, the library would be unavailable for backups, restorations and archiving. If Monday doesn't work for you, which day(s) in the next week or two would work for you? John

2 1

[CELS Sysadmins] Where's Ti?
by Leggett, Torrance I. 19 Jan '15

19 Jan '15

Tomorrow and Wednesday I'm in training all day in building 446.

1 0

[CELS Sysadmins] Where's Max?
by Adam (Max) Trefonides 19 Jan '15

19 Jan '15

Thursday, January 22: 1/2 day, I'll be at the lab in the morning and will be leaving at noon. Monday, January 26: 1/2 day I'll be at the lab in the morning and will be leaving at noon. Thursday, January 29: Full day, I'll be out of town and unreachable. -- Adam Max Trefonides CELS Systems maxadam(a)anl.gov

1 0

Re: [CELS Sysadmins] Test Userbase security fixes
by Leggett, Torrance I. 16 Jan '15

16 Jan '15

If you tried to test it before from certain networks, you may have been blocked. John found and corrected the blocks. If you could hammer on it and provide any bugs, that'd be most useful. I also pushed a DB optimization that should speed things up a good bit for larger queries. From: <Leggett>, Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>> Date: Wednesday, January 14, 2015 at 1:45 PM To: CELS Admins <sysadmins(a)lists.cels.anl.gov<mailto:[email protected]>> Cc: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>>, "Raffenetti, Kenneth J." <raffenet(a)mcs.anl.gov<mailto:[email protected]>> Subject: [CELS Sysadmins] Test Userbase security fixes The auditors that were hired to analyze the lab's web services found some security vulnerabilities (HTML injection and XSS). I think I've fixed these for anonymously provided user input which is the biggest concern, but because the changes could deeply affect data input and stored, I'd like a good deal of testing before I push it to production. Can I get everyone to give it a test and see if they can break things? Backend: https://sovereign-stage.mcs.anl.gov MCS Frontend https://mcs-stage.mcs.anl.gov JLSE Frontend https://jlse-stage.mcs.anl.gov KBase Frontend https://kb-stage.mcs.anl.gov

1 0

[CELS Sysadmins] where's Linda 1/16-21
by Linda Winkler 16 Jan '15

16 Jan '15

Working from home again on Fri in an attempt to feel better before traveling on Sun. Sun-Wed in Monterey for SC15 meeting. lw

1 0

[CELS Sysadmins] Where's max?
by Max Trefonides 16 Jan '15

16 Jan '15

Feeling like crap. Got no sleep. Going back to bed. Sick day please.

1 0

where's Linda 1/15
by Linda Winkler 15 Jan '15

15 Jan '15

Woke up with a funky throat. Going to WFM today to keep my germs isolated. lw

1 0

Re: [CELS Sysadmins] Test Userbase security fixes
by Leggett, Torrance I. 14 Jan '15

14 Jan '15

There may be ACLs in place for some networks and I've put in a conduit request for it. Also, any problems you find, screenshots and explanations sent to mcs-userbase(a)mcs.anl.gov is helpful. Thanks. From: <Leggett>, Ti Leggett <leggett(a)mcs.anl.gov<mailto:[email protected]>> Date: Wednesday, January 14, 2015 at 1:45 PM To: CELS Admins <sysadmins(a)lists.cels.anl.gov<mailto:[email protected]>> Cc: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>>, "Raffenetti, Kenneth J." <raffenet(a)mcs.anl.gov<mailto:[email protected]>> Subject: [CELS Sysadmins] Test Userbase security fixes The auditors that were hired to analyze the lab's web services found some security vulnerabilities (HTML injection and XSS). I think I've fixed these for anonymously provided user input which is the biggest concern, but because the changes could deeply affect data input and stored, I'd like a good deal of testing before I push it to production. Can I get everyone to give it a test and see if they can break things? Backend: https://sovereign-stage.mcs.anl.gov MCS Frontend https://mcs-stage.mcs.anl.gov JLSE Frontend https://jlse-stage.mcs.anl.gov KBase Frontend https://kb-stage.mcs.anl.gov

1 0