Introduction to the Command Line

Motivation

Many of us are new to using *nix systems
This can be a daunting system when thrown in
Using CLI is important for using our tools and firefighting

How to Use the Shell

Scroll through old commands: C-n, C-p
List your shell history: history
Search for old command: C-r
Move to the beginning of a line: C-a
Move to the end of a line: C-e

Basics

Echo

displays provided string or variable

$ echo "Hello World"
    Hello World

List Files

$ ls
    talk.html talk.org

$ ls -lah
    total 24
    drwxr-xr-x@  4 ngrayson  staff   136B Dec 15 15:13 .
    drwxr-xr-x@ 16 ngrayson  staff   544B Dec 15 14:49 ..
    -rw-r--r--@  1 ngrayson  staff   4.3K Dec 15 15:13 talk.html
    -rw-r--r--@  1 ngrayson  staff   845B Dec 15 15:13 talk.org

$ tree
    .
    ├── talk.html
    └── talk.org

    0 directories, 2 files

Note tree is likely not installed by default (brew install tree)

Change Directories

~ is your homedir
cd == cd ~
. is the current dir
.. is one dir ‘up’

$ cd /tmp
$ pwd
    /tmp
$ cd ~

Create Directories

$ mkdir test/testing
    mkdir: test: No such file or directory
$ mkdir -p test/testing

Create Files

touch, cp, mv, ln

$ touch file
$ cp file file2
$ mv file file1
$ ln -s file file1

Remove Files

$ rm testing/file
$ rm -rf testing

Show files

$ cat test/testing/file
    This
    is
    a
    file!
$ head -n 1
    This
$ tail -n 1
    file!

Wildcards

Can be used with most commands

$ cat test/testing/*
    This
    is
    a
    file!
    Hello World
$ ls /Users/*/banno/cookbooks/node-*

Docs

man <command>
–help, -h

Sudo

Runs a command as the super user (root)

$ sudo whoami
    Password:
    root

Environment

Variables that customize how you interact with programs

env

Displays all environment variables

$ env
    HOSTNAME=MacBook-Pro.local
    TERM=xterm

PATH

Contains paths to directories you want the system to look for executables
Setup in .bashrc, .bash_profile, or .zshrc

$ echo $PATH
    /Users/ngrayson/node_modules/.bin:/Users/ngrayson/.cask/bin:/usr/local/opt/ruby/bin:/Users/ngrayson/bin:/usr/local/share/python:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ngrayson

Export

Usually found in your shell profile
Sets the value of a variable for use later

$ export SECRET_CODE=abcdefgh
$ echo $SECRET_CODE
    abcdefgh

Reloading Your Environment

bash: source ~/.bashrc
zsh: run command: reload

Redirection

Pipe

Redirects all output from one command to another
Much like a function call
Can be chained together

$ ls -l|cut -d' ' -f1|grep -v total|sort|uniq -c|sort -nr
       4 -rw-r--r--@
       2 drwxr-xr-x@

Redirection Operator

> overwrites
>> appends

$ cat test/testing/file > test/newfile
$ cat test/newfile
    This
    is
    a
    file!
$ echo "Another line" >> test/newfile
$ cat test/newfile
    This
    is
    a
    file!
    Another line

Streams

stdin: stream of text going into a program
stdout: standard output of a program
stderr: typically used to display errors
Redirect stderr to stdout: 2>&1

tee

Reads from stdin and writes to stdout or a file

$ echo "This is a test" | tee
    This is a test
$ echo "This is a test" | tee test
    This is a test
$ cat test
    This is a test

Searching

Grep

Search a file or stream from pipe
Useful flags: -i, -v, -r

$ grep is test/testing/file
    This
    is
$ cat test/testing/file | grep -i this
    This

Grep Replacements

ack-grep
ag

$ ag This
    newfile
    1:This

    testing/file
    1:This

Find

This could be its own talk
Allows very specific searching and execution

$ find . -type d -name "*test*"
    ./test
    ./test/testing
$ find . -type f -name "*file*" -exec cat {} \;
    This
    is
    a
    file!
    Another line
    This
    is
    a
    file!
    Hello World

Locate

Uses system db of file

$ locate talk.org
    /Users/ngrayson/banno/internal-talks/adam-scala-jenkins-docker-big/talk.org
    /Users/ngrayson/banno/internal-talks/chef-nic-11-02-13/chef-talk.org

which

outputs the path to an executable on your PATH

$ which knife
    /Users/ngrayson/deploy/bin/knife
$ which -a knife
    /Users/ngrayson/deploy/bin/knife
    /usr/local/bin/knife

Processing Output

sort

$ cat letters
    b
    a
    d
    c
    a
    c
$ cat letters|sort
    a
    a
    b
    c
    c
    d

uniq

$ cat letters|sort |uniq
    a
    b
    c
    d
$ cat letters|sort |uniq -c |sort -nr
       2 c
       2 a
       1 d
       1 b

cut

$ cat words
    first second third
    1st 2nd 3rd
$ cat words|cut -d' ' -f2
    second
    2nd

rev

$ echo "abcde" |rev
    edcba

sed

Find and replace
There is a lot to this tool

$ echo "Hello World" |sed 's/World/Nic/'
    Hello Nic

wc

$ wc words
           2       6      31 words
$ wc -l words
           2 words

Permissions

chown

change owner and/or group of file

$ chown -R foo.bar test

chmod

u = user
g = group
o = other
r = read
w = write
x = execute

$ chmod u+x file
$ chmod ug-x file
$ chmod 755 file
$ chmod 644 file

Interacting with Processes

ps

lists running processes
aux, faux

$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  2.3  0.0  18172  3368 ?        Ss   22:26   0:00 bash
    root        17  0.0  0.0  15572  2096 ?        R+   22:26   0:00 ps aux

kill

kill -9 forces the kill if the process is hung

$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  1.1  0.0  18172  3340 ?        Ss   22:27   0:00 bash
    root        16  0.0  0.0   4348   656 ?        S    22:27   0:00 sleep 1000
    root        17  0.0  0.0  15572  2152 ?        R+   22:27   0:00 ps aux
$ kill 16
$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  0.1  0.0  18172  3344 ?        Ss   22:27   0:00 bash
    root        24  0.0  0.0  15572  2032 ?        R+   22:30   0:00 ps aux

top

Displays list of running process in an interactive way
Shift+m sorts by memory usage
Shift+p sorts by cpu usage
q exits

htop

more in depth version of top, not on all systems

&

Runs a process in the background

$ sleep 1000 &
    [1] 27187

Exit Status

When a command finishes it returns a number
This is placed in the $? variable
0 means the command completed successfully
More information here: http://tldp.org/LDP/abs/html/exitcodes.html

$ echo "test"
    test
$ echo $?
    0
$ cat notafile
    cat: notafile: No such file or directory
$ echo $?
    1

Networking

ping

Sends icmp packets at domain or ip

$ ping google.com
    PING google.com (74.125.207.113): 56 data bytes
    64 bytes from 74.125.207.113: icmp_seq=0 ttl=49 time=27.630 ms
    64 bytes from 74.125.207.113: icmp_seq=1 ttl=49 time=35.561 ms
    64 bytes from 74.125.207.113: icmp_seq=2 ttl=49 time=32.500 ms

host

Attempts to lookup a hostname for the ip provided

$ host 8.8.8.8
    8.8.8.8.in-addr.arpa domain name pointer google-public-dns-a.google.com.

dig

Looks up the ip associated with a domain name
+short displays only the ip address

$ dig banno.com

    ; <<>> DiG 9.8.3-P1 <<>> banno.com
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23178
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;banno.com.                     IN      A

    ;; ANSWER SECTION:
    banno.com.              9       IN      A       54.84.218.15

    ;; Query time: 146 msec
    ;; SERVER: 10.3.0.53#53(10.3.0.53)
    ;; WHEN: Tue Dec 16 10:20:13 2014
    ;; MSG SIZE  rcvd: 43
$ dig +short ns banno.com
    ns-1632.awsdns-12.co.uk.
    ns-692.awsdns-22.net.
    ns-347.awsdns-43.com.
    ns-1475.awsdns-56.org.

netstat

-lpn is most used to display listening ports
-lpn only shows the program name if you are root or sudoing
Does not work on mac

    root@mesos-master1:~# netstat -lpn
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1063/sshd
    tcp        0      0 10.10.1.33:5050         0.0.0.0:*               LISTEN      8201/mesos-master
    tcp6       0      0 :::8080                 :::*                    LISTEN      5761/docker-proxy
    tcp6       0      0 :::8081                 :::*                    LISTEN      10375/docker-proxy
    tcp6       0      0 :::22                   :::*                    LISTEN      1063/sshd
    udp        0      0 0.0.0.0:20436           0.0.0.0:*                           587/dhclient
    udp        0      0 0.0.0.0:68              0.0.0.0:*                           587/dhclient
    udp        0      0 172.17.42.1:123         0.0.0.0:*                           1543/ntpd
    udp        0      0 10.10.1.33:123          0.0.0.0:*                           1543/ntpd
    udp        0      0 127.0.0.1:123           0.0.0.0:*                           1543/ntpd
    udp        0      0 0.0.0.0:123             0.0.0.0:*                           1543/ntpd
    udp6       0      0 :::9816                 :::*                                587/dhclient
    udp6       0      0 :::123                  :::*                                1543/ntpd
    Active UNIX domain sockets (only servers)
    Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
    unix  2      [ ACC ]     SEQPACKET  LISTENING     7738     402/systemd-udevd   /run/udev/control
    unix  2      [ ACC ]     STREAM     LISTENING     8781     811/dbus-daemon     /var/run/dbus/system_bus_socket
    unix  2      [ ACC ]     STREAM     LISTENING     7171     1/init              @/com/ubuntu/upstart
    unix  2      [ ACC ]     STREAM     LISTENING     11940    1503/docker         /var/run/docker.sock
    unix  2      [ ACC ]     STREAM     LISTENING     9209     1069/acpid          /var/run/acpid.socket

ifconfig

Lists information about networking on your computer

$ ifconfig
    eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:0e
              inet addr:172.17.0.14  Bcast:0.0.0.0  Mask:255.255.0.0
              inet6 addr: fe80::42:acff:fe11:e/64 Scope:Link
              UP BROADCAST RUNNING  MTU:1500  Metric:1
              RX packets:3 errors:0 dropped:0 overruns:0 frame:0
              TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:258 (258.0 B)  TX bytes:418 (418.0 B)

    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Information Collection

df

Displays system wide disk usage
-h displays ‘human readable’

$ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    rootfs           96G   36G   55G  40% /
    none             96G   36G   55G  40% /
    tmpfs           2.0G     0  2.0G   0% /dev
    shm              64M     0   64M   0% /dev/shm
    /dev/sda1        96G   36G   55G  40% /etc/hosts
    tmpfs           2.0G     0  2.0G   0% /proc/kcore

du

Displays an estimate of file space usage

/var# du -sh *
    4.0K    backups
    2.7M    cache
    9.1M    lib
    4.0K    local
    0       lock
    292K    log
    4.0K    mail
    4.0K    opt
    0       run
    20K     spool
    4.0K    tmp

free

Shows current memory usage
-m display in megabytes
Does not work on Mac

free -m
             total       used       free     shared    buffers     cached
Mem:          3910       2391       1518        202        197       1760
-/+ buffers/cache:        433       3476
Swap:         1877          2       1874

w

Shows which users are logged in currently

$ w
    10:29  up 5 days, 18:12, 2 users, load averages: 2.24 1.99 1.89
    USER     TTY      FROM              LOGIN@  IDLE WHAT
    ngrayson console  -                 8:55    1:34 -

lastlog

Shows a list of users and when they have logged in

$ lastlog
    Username         Port     From             Latest
    root                                       **Never logged in**
    daemon                                     **Never logged in**
    bin                                        **Never logged in**
    sys                                        **Never logged in**
    sync                                       **Never logged in**
    games                                      **Never logged in**
    man                                        **Never logged in**
    lp                                         **Never logged in**
    mail                                       **Never logged in**
    news                                       **Never logged in**
    uucp                                       **Never logged in**
    proxy                                      **Never logged in**
    www-data                                   **Never logged in**
    backup                                     **Never logged in**
    list                                       **Never logged in**
    irc                                        **Never logged in**
    gnats                                      **Never logged in**
    nobody                                     **Never logged in**

Working with Files

tar

Archiving utility

$ tar cfvz test.tar.gz test
    a test
    a test/file1
    a test/file2
    a test/newfile
    a test/testing
    a test/testing/file
    a test/testing/file2
$ tar xfvz test.tar.gz
    x ./._test
    x test/
    x test/._file1
    x test/file1
    x test/._file2
    x test/file2
    x test/._newfile
    x test/newfile
    x test/._testing
    x test/testing/
    x test/testing/._file
    x test/testing/file
    x test/testing/._file2
    x test/testing/file2

zip & unzip

Creates or unzips files

$ zip test.zip test/*
      adding: test/file1 (stored 0%)
      adding: test/file2 (stored 0%)
      adding: test/newfile (stored 0%)
      adding: test/testing/ (stored 0%)
$ unzip test.zip
    Archive:  test.zip
     extracting: test/file1
     extracting: test/file2
     extracting: test/newfile
       creating: test/testing/

rsync

Used to sync two directories
Can go over ssh

$ rsync -plarv test/ test2/
    building file list ... done
    created directory test2
    ./
    file1
    file2
    newfile
    testing/

    sent 315 bytes  received 98 bytes  826.00 bytes/sec
    total size is 29  speedup is 0.07

scp

Uses ssh to copy a file to another computer

$ scp test.tar.gz stagingapps.foo.com:/home/ngrayson/
    test.tar.gz                                                                                                                                   100%  906     0.9KB/s   00:00

Customizing Your Shell

alias

Create reusable shortcut
Place in your profile file to always have them

$ ls
    talk.html   talk.org    test        test.tar.gz test.zip    test2
$ alias ll="ls -lah"
$ ll
    total 96
    drwxr-xr-x@  8 ngrayson  staff   272B Dec 16 10:52 .
    drwxr-xr-x@ 16 ngrayson  staff   544B Dec 16 10:52 ..
    -rw-r--r--@  1 ngrayson  staff    23K Dec 16 10:52 talk.html
    -rw-r--r--@  1 ngrayson  staff    14K Dec 16 10:52 talk.org
    drwxr-xr-x@  6 ngrayson  staff   204B Dec 16 10:48 test
    -rw-r--r--@  1 ngrayson  staff   906B Dec 16 10:47 test.tar.gz
    -rw-r--r--@  1 ngrayson  staff   653B Dec 16 10:48 test.zip
    drwxr-xr-x@  6 ngrayson  staff   204B Dec 16 10:48 test2

zsh + oh-my-zsh

https://github.com/robbyrussell/oh-my-zsh

Questions?

If something here is out of date or you have a suggestion, send me an email.