Introduction to the Command Line

Motivation

Many of us are new to using *nix systems
This can be a daunting system when thrown in
Using CLI is important for using our tools and firefighting

How to Use the Shell

Scroll through old commands: C-n, C-p
List your shell history: history
Search for old command: C-r
Move to the beginning of a line: C-a
Move to the end of a line: C-e

Basics

Echo

displays provided string or variable

$ echo "Hello World"
    Hello World

List Files

$ ls
    talk.html talk.org

$ ls -lah
    total 24
    drwxr-xr-x@  4 ngrayson  staff   136B Dec 15 15:13 .
    drwxr-xr-x@ 16 ngrayson  staff   544B Dec 15 14:49 ..
    -rw-r--r--@  1 ngrayson  staff   4.3K Dec 15 15:13 talk.html
    -rw-r--r--@  1 ngrayson  staff   845B Dec 15 15:13 talk.org

$ tree
    .
    ├── talk.html
    └── talk.org

    0 directories, 2 files

Note tree is likely not installed by default (brew install tree)

Change Directories

~ is your homedir
`cd` == `cd ~`
. is the current dir
.. is one dir ‘up’

$ cd /tmp
$ pwd
    /tmp
$ cd ~

Create Directories

$ mkdir test/testing
    mkdir: test: No such file or directory
$ mkdir -p test/testing

Create Files

touch, cp, mv, ln

$ touch file
$ cp file file2
$ mv file file1
$ ln -s file file1

Remove Files

$ rm testing/file
$ rm -rf testing

Show files

$ cat test/testing/file
    This
    is
    a
    file!
$ head -n 1
    This
$ tail -n 1
    file!

Wildcards

Can be used with most commands

$ cat test/testing/*
    This
    is
    a
    file!
    Hello World
$ ls /Users/*/banno/cookbooks/node-*

Docs

man
–help, -h

Sudo

Runs a command as the super user (root)

$ sudo whoami
    Password:
    root

Environment

Variables that customize how you interact with programs

env

Displays all environment variables

$ env
    HOSTNAME=MacBook-Pro.local
    TERM=xterm

PATH

Contains paths to directories you want the system to look for executables
Setup in .bashrc, .bash_profile, or .zshrc

$ echo $PATH
    /Users/ngrayson/node_modules/.bin:/Users/ngrayson/.cask/bin:/usr/local/opt/ruby/bin:/Users/ngrayson/bin:/usr/local/share/python:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ngrayson

Export

Usually found in your shell profile
Sets the value of a variable for use later

$ export SECRET_CODE=abcdefgh
$ echo $SECRET_CODE
    abcdefgh

Reloading Your Environment

bash: source ~/.bashrc
zsh: run command: reload

Redirection

Pipe

Redirects all output from one command to another
Much like a function call
Can be chained together

$ ls -l|cut -d' ' -f1|grep -v total|sort|uniq -c|sort -nr
       4 -rw-r--r--@
       2 drwxr-xr-x@

Redirection Operator

> overwrites
>> appends

$ cat test/testing/file > test/newfile
$ cat test/newfile
    This
    is
    a
    file!
$ echo "Another line" >> test/newfile
$ cat test/newfile
    This
    is
    a
    file!
    Another line

Streams

stdin: stream of text going into a program
stdout: standard output of a program
stderr: typically used to display errors
Redirect stderr to stdout: 2>&1

tee

Reads from stdin and writes to stdout or a file

$ echo "This is a test" | tee
    This is a test
$ echo "This is a test" | tee test
    This is a test
$ cat test
    This is a test

Searching

Grep

Search a file or stream from pipe
Useful flags: -i, -v, -r

$ grep is test/testing/file
    This
    is
$ cat test/testing/file | grep -i this
    This

Grep Replacements

ack-grep
ag

$ ag This
    newfile
    1:This

    testing/file
    1:This

Find

This could be it’s own talk
Allow very specific searching and execution

$ find . -type d -name "*test*"
    ./test
    ./test/testing
$ find . -type f -name "*file*" -exec cat {} \;
    This
    is
    a
    file!
    Another line
    This
    is
    a
    file!
    Hello World

Locate

Uses system db of file

$ locate talk.org
    /Users/ngrayson/banno/internal-talks/adam-scala-jenkins-docker-big/talk.org
    /Users/ngrayson/banno/internal-talks/chef-nic-11-02-13/chef-talk.org

which

outputs the path to an execuctable on your PATH

$ which knife
    /Users/ngrayson/deploy/bin/knife
$ which -a knife
    /Users/ngraysondeploy/bin/knife
    /usr/local/bin/knife

Processing Output

sort

$ cat letters
    b
    a
    d
    c
    a
    c
$ cat letters|sort
    a
    a
    b
    c
    c
    d

uniq

$ cat letters|sort |uniq
    a
    b
    c
    d
$ cat letters|sort |uniq -c |sort -nr
       2 c
       2 a
       1 d
       1 b

cut

$ cat words
    first second third
    1st 2nd 3rd
$ cat words|cut -d' ' -f2
    second
    2nd

rev

$ echo "abcde" |rev
    edcba

sed

Find and replace
There is a lot to this tool

$ echo "Hello World" |sed 's/World/Nic/'
    Hello Nic

wc

$ wc words
           2       6      31 words
$ wc -l words
           2 words

Permissions

chown

change owner and/or group of file

$ chown -R foo.bar test

chmod

u = user
g = group
o = other
r = read
w = write
x = execute

$ chmod u+x file
$ chmod ug-x file
$ chmod 755 file
$ chmod 644 file

Interacting with Processes

ps

lists running processes
aux, faux

$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  2.3  0.0  18172  3368 ?        Ss   22:26   0:00 bash
    root        17  0.0  0.0  15572  2096 ?        R+   22:26   0:00 ps aux

kill

kill -9 forces the kill if the process is hung

$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  1.1  0.0  18172  3340 ?        Ss   22:27   0:00 bash
    root        16  0.0  0.0   4348   656 ?        S    22:27   0:00 sleep 1000
    root        17  0.0  0.0  15572  2152 ?        R+   22:27   0:00 ps aux
$ kill 16
$ ps aux
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root         1  0.1  0.0  18172  3344 ?        Ss   22:27   0:00 bash
    root        24  0.0  0.0  15572  2032 ?        R+   22:30   0:00 ps aux

top

Displays list of running process in an interactive way
Shift+m sorts by memory usage
Shift+p sorts by cpu usage
q exits

htop

more in depth version of top, not on all systems

&

Runs a process in the background

$ sleep 1000 &
    [1] 27187

Exit Status

When a command finishes it returns a number
This is placed in the $? variable
0 means the command completed succuessfully
More information here: http://tldp.org/LDP/abs/html/exitcodes.html

$ echo "test"
    test
$ echo $?
    0
$ cat notafile
    cat: notafile: No such file or directory
$ echo $?
    1

Networking

ping

Sends icmp packets at domain or ip

$ ping google.com
    PING google.com (74.125.207.113): 56 data bytes
    64 bytes from 74.125.207.113: icmp_seq=0 ttl=49 time=27.630 ms
    64 bytes from 74.125.207.113: icmp_seq=1 ttl=49 time=35.561 ms
    64 bytes from 74.125.207.113: icmp_seq=2 ttl=49 time=32.500 ms

host

Attempts to lookup a hostname for the ip provided

$ host 8.8.8.8
    8.8.8.8.in-addr.arpa domain name pointer google-public-dns-a.google.com.

dig

Looks up the ip associated with a domain name
+short displays only the ip address

$ dig banno.com

    ; <<>> DiG 9.8.3-P1 <<>> banno.com
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23178
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;banno.com.                     IN      A

    ;; ANSWER SECTION:
    banno.com.              9       IN      A       54.84.218.15

    ;; Query time: 146 msec
    ;; SERVER: 10.3.0.53#53(10.3.0.53)
    ;; WHEN: Tue Dec 16 10:20:13 2014
    ;; MSG SIZE  rcvd: 43
$ dig +short ns banno.com
    ns-1632.awsdns-12.co.uk.
    ns-692.awsdns-22.net.
    ns-347.awsdns-43.com.
    ns-1475.awsdns-56.org.

netstat

-lpn is most used to display listening ports
-lpn only shows the program name if you are root or sudoing
Does not work on mac

    root@mesos-master1:~# netstat -lpn
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1063/sshd
    tcp        0      0 10.10.1.33:5050         0.0.0.0:*               LISTEN      8201/mesos-master
    tcp6       0      0 :::8080                 :::*                    LISTEN      5761/docker-proxy
    tcp6       0      0 :::8081                 :::*                    LISTEN      10375/docker-proxy
    tcp6       0      0 :::22                   :::*                    LISTEN      1063/sshd
    udp        0      0 0.0.0.0:20436           0.0.0.0:*                           587/dhclient
    udp        0      0 0.0.0.0:68              0.0.0.0:*                           587/dhclient
    udp        0      0 172.17.42.1:123         0.0.0.0:*                           1543/ntpd
    udp        0      0 10.10.1.33:123          0.0.0.0:*                           1543/ntpd
    udp        0      0 127.0.0.1:123           0.0.0.0:*                           1543/ntpd
    udp        0      0 0.0.0.0:123             0.0.0.0:*                           1543/ntpd
    udp6       0      0 :::9816                 :::*                                587/dhclient
    udp6       0      0 :::123                  :::*                                1543/ntpd
    Active UNIX domain sockets (only servers)
    Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
    unix  2      [ ACC ]     SEQPACKET  LISTENING     7738     402/systemd-udevd   /run/udev/control
    unix  2      [ ACC ]     STREAM     LISTENING     8781     811/dbus-daemon     /var/run/dbus/system_bus_socket
    unix  2      [ ACC ]     STREAM     LISTENING     7171     1/init              @/com/ubuntu/upstart
    unix  2      [ ACC ]     STREAM     LISTENING     11940    1503/docker         /var/run/docker.sock
    unix  2      [ ACC ]     STREAM     LISTENING     9209     1069/acpid          /var/run/acpid.socket

ifconfig

Lists information about networking on your computer

$ ifconfig
    eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:0e
              inet addr:172.17.0.14  Bcast:0.0.0.0  Mask:255.255.0.0
              inet6 addr: fe80::42:acff:fe11:e/64 Scope:Link
              UP BROADCAST RUNNING  MTU:1500  Metric:1
              RX packets:3 errors:0 dropped:0 overruns:0 frame:0
              TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:258 (258.0 B)  TX bytes:418 (418.0 B)

    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Information Collection

df

Displays system wide disk usage
-h displays ‘human readable’

$ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    rootfs           96G   36G   55G  40% /
    none             96G   36G   55G  40% /
    tmpfs           2.0G     0  2.0G   0% /dev
    shm              64M     0   64M   0% /dev/shm
    /dev/sda1        96G   36G   55G  40% /etc/hosts
    tmpfs           2.0G     0  2.0G   0% /proc/kcore

du

Displays an estimate of file space usage

    /var# du -sh *
    4.0K    backups
    2.7M    cache
    9.1M    lib
    4.0K    local
    0       lock
    292K    log
    4.0K    mail
    4.0K    opt
    0       run
    20K     spool
    4.0K    tmp

free

Shows current memory usage
-m display in megabytes
Does not work on Mac

    free -m
                 total       used       free     shared    buffers     cached
    Mem:          3910       2391       1518        202        197       1760
    -/+ buffers/cache:        433       3476
    Swap:         1877          2       1874

w

Shows which users are logged in currently

$ w
    10:29  up 5 days, 18:12, 2 users, load averages: 2.24 1.99 1.89
    USER     TTY      FROM              LOGIN@  IDLE WHAT
    ngrayson console  -                 8:55    1:34 -

lastlog

Shows a list of users and when they have logged in

$ lastlog
    Username         Port     From             Latest
    root                                       **Never logged in**
    daemon                                     **Never logged in**
    bin                                        **Never logged in**
    sys                                        **Never logged in**
    sync                                       **Never logged in**
    games                                      **Never logged in**
    man                                        **Never logged in**
    lp                                         **Never logged in**
    mail                                       **Never logged in**
    news                                       **Never logged in**
    uucp                                       **Never logged in**
    proxy                                      **Never logged in**
    www-data                                   **Never logged in**
    backup                                     **Never logged in**
    list                                       **Never logged in**
    irc                                        **Never logged in**
    gnats                                      **Never logged in**
    nobody                                     **Never logged in**

Working with Files

tar

Archiving utility

$ tar cfvz test.tar.gz test
    a test
    a test/file1
    a test/file2
    a test/newfile
    a test/testing
    a test/testing/file
    a test/testing/file2
$ tar xfvz test.tar.gz
    x ./._test
    x test/
    x test/._file1
    x test/file1
    x test/._file2
    x test/file2
    x test/._newfile
    x test/newfile
    x test/._testing
    x test/testing/
    x test/testing/._file
    x test/testing/file
    x test/testing/._file2
    x test/testing/file2

zip & unzip

Created or unzips files

$ zip test.zip test/*
      adding: test/file1 (stored 0%)
      adding: test/file2 (stored 0%)
      adding: test/newfile (stored 0%)
      adding: test/testing/ (stored 0%)
$ unzip test.zip
    Archive:  test.zip
     extracting: test/file1
     extracting: test/file2
     extracting: test/newfile
       creating: test/testing/

rsync

Used to sync two directory
Can go over ssh

$ rsync -plarv test/ test2/
    building file list ... done
    created directory test2
    ./
    file1
    file2
    newfile
    testing/

    sent 315 bytes  received 98 bytes  826.00 bytes/sec
    total size is 29  speedup is 0.07

scp

Uses ssh to copy a file to another computer

$ scp test.tar.gz stagingapps.foo.com:/home/ngrayson/
    test.tar.gz                                                                                                                                   100%  906     0.9KB/s   00:00

Customizing Your Shell

alias

Create resusable shortcut
Place in your profile file to always have them

$ ls
    talk.html   talk.org    test        test.tar.gz test.zip    test2
$ alias ll="ls -lah"
$ ll
    total 96
    drwxr-xr-x@  8 ngrayson  staff   272B Dec 16 10:52 .
    drwxr-xr-x@ 16 ngrayson  staff   544B Dec 15 15:23 ..
    -rw-r--r--@  1 ngrayson  staff    23K Dec 16 10:52 talk.html
    -rw-r--r--@  1 ngrayson  staff    14K Dec 16 10:52 talk.org
    drwxr-xr-x@  6 ngrayson  staff   204B Dec 16 10:48 test
    -rw-r--r--@  1 ngrayson  staff   906B Dec 16 10:47 test.tar.gz
    -rw-r--r--@  1 ngrayson  staff   653B Dec 16 10:48 test.zip
    drwxr-xr-x@  6 ngrayson  staff   204B Dec 16 10:48 test2

zsh + oh_my_zsh

https://github.com/robbyrussell/oh-my-zsh

Nic Grayson

Introduction to the Command Line

Motivation

How to Use the Shell

Basics

Echo

List Files

Change Directories

Create Directories

Create Files

Remove Files

Show files

Wildcards

Docs

Sudo

Environment

env

PATH

Export

Reloading Your Environment

Redirection

Pipe

Redirection Operator

Streams

tee

Searching

Grep

Grep Replacements

Find

Locate

which

Processing Output

sort

uniq

cut

rev

sed

wc

Permissions

chown

chmod

Interacting with Processes

ps

kill

top

htop

&

Exit Status

Networking

ping

host

dig

netstat

ifconfig

Information Collection

df

du

free

w

lastlog

Working with Files

tar

zip & unzip

rsync

scp

Customizing Your Shell

alias

zsh + oh_my_zsh

Questions?

You might also enjoy (View all posts)