logo

Shell Cheatsheet - Scripts

Loop

for S in FOO BAR BAZ
do
    ...
done

Loop in a sequence:

for i in {0..10}

With a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
for i in {0..10..2}

The other syntax to generate a sequence:

for i in $(seq 0 2 10)

Environment Variables and Shell Variables

# List all variables
$ env
$ printenv

# Show A Specific Variable
$ env | grep TERM
xterm-256color

$ echo $TERM
xterm-256color

# Remove A Variable
$ unset $VAR_NAME

env vs printenv: printenv is equivelant to env when printing env vars; env can also be used to set env vars.

Set Environment Variable

$ x="hello" y="world" bash -c 'echo $x $y'
hello world

bash -c: "commands are read from the first non-option argument command_string. If there are arguments after the command_string, the first argument is assigned to 0 a n d a n y r e m a i n i n g a r g u m e n t s a r e a s s i g n e d t o t h e p o s i t i o n a l p a r a m e t e r s . T h e a s s i g n m e n t t o 0 and any remaining arguments are assigned to the positional parameters. The assignment to 0 sets the name of the shell, which is used in warning and error messages."

List of Built-in Variables

  • HOME: Home directory of the user. Default is /home/<your_username>/
  • MAIL - Contains the path to the location where mail addressed to the user is stored.
  • IFS - Internal Field Separator. Contains a string of characters which are used as word separators in the command line. The string normally consists of the space, tab and the newline characters. To see them you will have to do an octal dump: $ echo $IFS | od -bc
  • PS1 and PS2 - Primary and secondary prompts in bash. PS1 is set to $ by default and PS2 is set to > . To see the secondary prompt: $ ls |
  • USER - User login name.
  • TERM - indicates the terminal type being used. This should be set correctly for editors like Vim to work correctly.
  • SHELL - Determines the type of shell that the user sees on logging in. Default should be /bin/bash on most Linux distros; macOS defaults to /bin/zsh.
  • LOGNAME - login name
  • PATH: where to look for commands

Check and change your shell

$ echo $SHELL
$ echo $0

Change your default shell (e.g. to zsh):

$ chsh -s /usr/bin/zsh

env variable vs shell variable

  • Environment variables are NOT global/system-wide. They are available in current shell and all child processes.
  • Shell variables are private to the currently running shell, and they are NOT exported (passed on) to any child processes.

To make a shell variable available as an environment variable, use export VARNAME

To export or not to export?

  • If the var is expanded right here in the command / script, it can go without export

    PROJECT=foo
mkdir -p tmp/$PROJECT

  • If the var is used in subprocesses, use export; e.g. if your build tool / script allow you to override env var like XXX_VERSION, you can export XXX_VERSION in the main script.

Alias

$ alias sayhello='echo hello'
$ sayhello
hello

To show colors by file extension

$ ls -G

To make it default, add this to ~/.bash_profile

alias ls="ls -G"

Wildcards

  • *: matches any characters
  • ?: matches one and only one character
  • [abc]: matches a or b or c
  • [!abc]: matches any characters other than a or b or c
  • {abc, def}: matches abc or def
  • {1..5}: generates the sequence 1 2 3 4 5

Input/Output

A shell opens 3 standard IO streams:

  • 0: standard input stdin
  • 1: standard output stdout
  • 2: standard error stderr

They are basically file handles ("Everything is a file").

Redirect

  • < redirects stdin, e.g. $ sort < /etc/group

  • > redirects stdout, e.g. $ echo 'hello' > hello.txt

  • 2> redirects stderr, e.g. 2> will redirect stderr to errors.txt:

    $ find / -name term 2> errors.txt

  • 2>&1 assigns stream 2 to 1, so all stderr is also showing in stdout, e.g.

    $ find / -name term 2>&1 > all.txt

$ (do whatever) > /home/ubuntu/`date +"%Y%m%d"`.log 2>&1

  • | (a pipe) transfers the stdout as the input of next segment

Capture the output of command line

Use backticks:

var=`command`

Use $():

var=$(command)

yes or no

Some commands/programs will pause to ask you y or N. To Answer that automatically, use yes or yes n for no

$ yes | xxxx

$ yes n | xxxx

Brackets

Name Other Name
[] Brackets
() Round Brackets parentheses
{} Curly brackets braces
<> Angle brackets chevrons

In Bash, test and [ are builtins. The double bracket enables additional functionality. For example, you can use && and || instead of -a and -o and there's a regular expression matching operator =~.

The double bracket enables additional functionality. For example, you can use && and || instead of -a and -o and there's a regular expression matching operator =~.

Shell Parameter Expansion

The braces, in addition to delimiting a variable name are used for parameter expansion so you can do things like:

Truncate the contents of a variable

$ var="abcde"; echo ${var%d*}
abc

Make substitutions similar to sed

$ var="abcde"; echo ${var/de/12}
abc12

Use a default value

$ default="hello"; unset var; echo ${var:-$default}
hello

and several more

Also, brace expansions create lists of strings which are typically iterated over in loops:

$ echo f{oo,ee,a}d
food feed fad

$ mv error.log{,.OLD}
(error.log is renamed to error.log.OLD because the brace expression
expands to "mv error.log error.log.OLD")

$ for num in {000..2}; do echo "$num"; done
000
001
002

$ echo {00..8..2}
00 02 04 06 08

$ echo {D..T..4}
D H L P T

Note that the leading zero and increment features weren't available before Bash 4.

Double parentheses

Double parentheses are used for arithmetic operations:

((a++))

((meaning = 42))

for ((i=0; i<10; i++))

echo $((a + b + (14 * c)))

and they enable you to omit the dollar signs on integer and array variables and include spaces around operators for readability.

Single brackets are also used for array indices:

array[4]="hello"

element=${array[index]}

Curly brace are required for (most/all?) array references on the right hand side.

Parentheses are also used for subshells. And that they are used to create arrays.

array=(1 2 3)
echo ${array[1]}
2

[ vs [[: http://mywiki.wooledge.org/BashFAQ/031

Quote

$'string': a special kind of single quotes.

  • replace backslash-escaped characters (e.g. \t for tab)
  • then function as a single quoted string.

e.g.

$ echo $'Name\tAge\nBob\t24\nMary\t36'
Name    Age
Bob     24
Mary    36

:- - Parameter substitution

:-: e.g. ${a:-b}, if a is unset or null, set to b.

# if `REGISTRY_HOST` not set, set to `HOST_IP`
export REGISTRY_HOST=${REGISTRY_HOST:-${HOST_IP}}

Shell - :? vs ?

  • ${parameter:?err}: if parameter is null or unset, print error message err
  • ${parameter?err}: if parameter is unset, print error message err (does NOT check null)
# a is set, not null
$ a=hello
$ echo ${a:?something went wrong}
hello

# b is not set
$ echo ${b:?something went wrong}
zsh: b: something went wrong

$ echo ${b?something went wrong}
zsh: b: something went wrong

# c is null, only :? shows error
$ echo ${c:?something went wrong}
zsh: c: something went wrong

$ echo ${c?something went wrong}
# no error

How to use param in alias?

alias does not support params; use a function instead.

myfunction() {
    echo $1
}

myfunction hello

Trap

tempfile=/tmp/tmpdata
function cleanup() {
    rm -f $tempfile
}
trap cleanup EXIT

cleanup will be called upon exit. Equivalent to a defer statement in Swift or Go.

Shebang

The #! is called a shebang. Scripts will execute using the interpreter specified on a first line.

Make Shell Script Portable

If you use the POSIX shell /bin/sh, just add #!/bin/sh to the top of your script. The location is standardized.

If you use other shells, e.g. bash, use #!/usr/bin/env bash instead of #!/bin/bash as shebang. Because bash is not always in the same location. If you have multiple versions of a shell installed, use env will make sure the first executable in the PATH will be used.

Bash Script

bash syntax: https://devmanual.gentoo.org/tools-reference/bash/index.html

Evaluate numbers

$((i-1))

Check if exists

if [ -d foo ]; then
    mkdir foo
fi

Skeleton of a bash script

#!/bin/bash

usage="Usage: server.sh (start|stop)"

# command is 'start' or 'stop'
command=$1

case $command in

    (start)
        ...
        ;;

    (stop)
        ...
        ;;

    (*)
        echo $usage
        exit 1
        ;;
esac
logo
PRIVACY POLICYABOUT中文