Hey UNIX geeks

Discussion in 'OT Technology' started by trouphaz, Nov 7, 2008.

  1. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    So, what is your preferred way of doing repetitive tasks from the shell? There are a bunch of ways to take stdout and run tasks against the output. Do you prefer for loops, while loops, xargs, find with -exec (if you are working on files), what? I personally use for, while and xargs and completely avoid find with -exec.

    I use the following a lot:

    for X in hostname1 hostname2 ... hostnameN
    do
    tar cvf - /path/to/files | ssh $X "tar xvf -"
    done

    vxdisk list | grep error | cut -d"s" -f1 | while read DISK
    do
    format $DISK <<EOF
    l
    y
    EOF
    done

    find . -mtime +15 ! -name "*.gz" | xargs -i gzip {}




    Do you have preferred ways of doing repetitive tasks?
     
  2. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    i just run this once a day:

    [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo "You live"
     
  3. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    I always break out the perl.
     
  4. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    that's a good point. i don't know why people still do shell scripting when there is a language as powerful as perl just waiting to be used.
     
  5. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    because shell scripts are still plenty powerful without having to learn a new language. shell scripts behave like your shell behaves. while it may be more cumbersome at times, at least it is predictable.

    that and laziness. laziness factors in quite heavily.
     
  6. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    shell scripting is a language of it's own. if you know how to use your shell but have never created a shell script you'd still have to learn a limited language. minus whale learn perl since it can do everything you could do with shell scripting and more?
     
  7. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    well, yes and no. see, you can start shell scripting basics by just taking commands that you run manually and putting them in a file. now, you can run that script so you don't have to keep retyping those commands or whatever. then, you realize one day commands 3-10 fail because command 2 didn't finish properly, so you start to learn about error checking. then, you realize you want to be able to use that same script on another host, so you start to figure out making the script more dynamic by pulling in such info as IP addresses and hostnames on the fly. then, you want to make the script work better with slightly different configurations so you learn about passing arguments to the script. then you realize that you keep using one piece of code over and over again, so you start working on building portable functions that can be dropped into other scripts.

    so, it isn't a matter of learning everything new at one shot. my shell scripts now are very advanced compared to my originals which were just a way of me simplifying repetitive tasks.


    but, getting back to the original question, i'm referring to simple, but repetitive tasks that are done from the command line, not really shell scripts per se. you've just assigned 40 new luns to a Solaris host and you have to put a label on all of them. Perl just doesn't seem like the right choice. you've built a 5 server cluster and realize you have to update a file on all of them. you have 100 servers in your datacenter and need to connect to each one of them to verify they can each properly resolve a specific address.

    for X in `cat host.list`
    do
    ssh $X "ping 192.168.1.1 2"
    done

    cat host.list | while read X
    do
    ssh $X "ping 192.168.1.1 2"
    done

    cat host.list | xargs -i ssh {} "ping 192.168.1.1 2"


    do you ever find yourself doing stuff like this? do you have a preferred way of doing it or is that just me?
     
  8. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    pfft, that's not scripting, that's a batch file.

    then you realize you need more than a batch file, so you start learning perl! :mamoru: :p
     
  9. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    well, yes. that's what i'm saying. you can get your peepee wet with shell scripts by starting with batch files and slowly work your way up to error checking, recursion, functions, user input, command line options, etc. so, it is very functional from day 1. perl, on the other hand, has limited function for an SA when you first start learning. then, quite a lot of what i do ultimately calls unix commands which is easier with a shell script.

    EDIT: by the way, this is in no way a claim that shell scripting is better than perl scripting. it is just more straight forward as an SA to get into shell scripting. so, i'm not looking to rehash any argument about perl or anythign like that. :)
     
  10. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    i wouldn't be so sure, running a unix command in perl is as easy as putting `'s around the command.

    i'll admit, i am a shell scripting nub, but i feel that is mostly the case because anytime i've needed to do anything that would require a shell script i've easily accomplished it with a perl script. so i'm kind of in the opposite situation, i know perl, why learn shell scripting?
     
  11. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    I don't use bash because I forgot it. So I use perl. bash is often more powerful for command line tasks.
     
  12. crontab

    crontab (uid = 0)

    Joined:
    Nov 14, 2000
    Messages:
    23,448
    Likes Received:
    12
    same as you, mostly for loops and xargs. some while loops, never find -exec, always find | into xargs, much faster.

    i don't do for i in hostname1 hostname2 hostname3...

    if it's a list, i cat it obviously, but it's a number that increments i do

    for i in `seq 1 10`
    do
    tar cvf - /path/to/files | ssh hostname$i "tar xvf -"
    done
     
  13. CodeX

    CodeX Guest

    Talk about a trivial question.
     
  14. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    ooh, what is that "seq 1 10" business? i'm guessing that it loops once for each number 1-10. i never saw that before. i usually do something like:

    typeset -i Y=1
    while [ $Y -le 10 ]
    do
    tar cvf - /path/to/files | ssh hostname$i "tar xvf -"
    Y=$Y+1
    done

    and that is because i usually forget the $(($Y+1)) notation or whatever it is for arithmetic.
     
  15. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    by the way, go troll elsewhere you useless asshat.
     
  16. CodeX

    CodeX Guest

    Tell the people I help with their programming problems I'm a useless asshat, don't ever see you jumping in to help anyone.
     
  17. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    yawn.
     
  18. JustJeff

    JustJeff www.youtube.com/thisisjustjeff

    Joined:
    Oct 30, 2006
    Messages:
    1,651
    Likes Received:
    0
    Location:
    Long Island // Virginia Tech
    Man, the harshness...

    Perl is the way to go, no doubt... but I haven't used perl in almost a year now. I'm pretty sure I've forgotten everything.

    But that is why you have books on that kind of stuff!
     
  19. piratepenguin

    piratepenguin New Member

    Joined:
    Jun 18, 2006
    Messages:
    1,067
    Likes Received:
    0
    Location:
    Ireland
  20. Limp_Brisket

    Limp_Brisket New Member

    Joined:
    Jan 2, 2006
    Messages:
    48,422
    Likes Received:
    0
    Location:
    Utah
    my professor calls python "pseudo code". i think he means it in a good way though :mamoru:
     
  21. trouphaz

    trouphaz New Member

    Joined:
    Sep 22, 2003
    Messages:
    2,666
    Likes Received:
    0
    ah, so seq is not a standard Solaris command apparently.
     
  22. crontab

    crontab (uid = 0)

    Joined:
    Nov 14, 2000
    Messages:
    23,448
    Likes Received:
    12
    ah no, linux. i find myself doing most of my scripting under linux. i find the gnu variants more useful than the solaris variants. even when i have data generated on sun boxes, i output the data to some nas share and then work with from another linux machine... even when i don't need to....
     
  23. piratepenguin

    piratepenguin New Member

    Joined:
    Jun 18, 2006
    Messages:
    1,067
    Likes Received:
    0
    Location:
    Ireland
    Honestly am I mad if I answered Python?

    I don't know Python but I'm learning. The main reason I'm learning it and not Perl is because I want it for web-work (Django) and different projects I want to contribute to, while ALSO I can use it for scripts and the like.
     
  24. ranthal

    ranthal i did what?

    Joined:
    Feb 8, 2007
    Messages:
    772
    Likes Received:
    0
    Location:
    Los Angeles
    damn i swear anytime i see any UNIX or linux discussions shit has already hit the fan by the end of the first page that i can't even tell where the thread went by skimming it
     
  25. Peyomp

    Peyomp New Member

    Joined:
    Jan 11, 2002
    Messages:
    14,017
    Likes Received:
    0
    Except this one.
     

Share This Page