By Shawn Wilson
I assume some prior Unix command line experience
I keep running into "bash scripts" that look like old style shell scripts - capitalized variable names, tons of subshells piping all over the place, etc. This does not need to be the case - bash scripts can be written with some elegance
I hope this talk shows some mechanisms you can use to write nicer bash scripts
While some help docs are incomplete, (besides the 3100 line
man page) these are the best resources on bash (paid or
free):
These documents are generally about a page long and do not
go through a pager so may scroll.
$ help help | head -4
help: help [-dms] [pattern ...]
Display information about builtin commands.
Displays brief summaries of builtin commands. If PATTERN is
$ help [ | head -4
[: [ arg... ]
Evaluate conditional expression.
This is a synonym for the "test" builtin, but the last argument must
$ help [[ | head -4
[[ ... ]]: [[ expression ]]
Execute conditional command.
Returns a status of 0 or 1 depending on the evaluation of the conditional
$ help declare | head -4
declare: declare [-aAfFgilnrtux] [-p] [name[=value] ...]
Set variable values and attributes.
Declare variables and give them attributes. If no NAMEs are given,
$ help set | head -4
set: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
Set or unset values of shell options and positional parameters.
Change the value of shell attributes and positional parameters, or
$ help enable | head -4
enable: enable [-a] [-dnps] [-f filename] [name ...]
Enable and disable shell builtins.
Enables and disables builtin shell commands. Disabling allows you to
$ help hash | head -4
hash: hash [-lr] [-p pathname] [-dt] [name ...]
Remember or display program locations.
Determine and remember the full pathname of each command NAME. If
Bash has an order of operations for when and where to look for stuff:
So, for instance the test operator ([) is actually a command as well as a built-in:
$ type [
[ is a shell builtin
$ whereis [
[: /usr/bin/[ /usr/share/man/man1/[.1.gz
{0..5}
$(seq 0 5)
DON'T DO THIS
for i in $(seq 0 5); do ...; done
EVER
But maybe just this one time?
NO
Maybe you're still listening ant want an alternative?
for i in {0..5}; do ...; done
Now what is bash doing?
The for loop loops over "words" (which can be a bash array or the english definition of a "word" - space/tab/newline separated words)
The while loop uses read which has a bit more going on
In either case "words" can be combined into a word with quotes
Examples of the two commands:
$ echo {0..3}
0 1 2 3
$ seq 0 3
0
1
2
3
With bash expansion, any command that accepts multiple parameters may be used for cheap fuzzing:
dig computer{1..5}.example.com
But really, why stop there?
$ echo foo{0..1}{0..1}
foo00 foo01 foo10 foo11
And if you really want to fuzz stuff (I'm guessing there's a better way)
for i in {0..255}; do eval "echo -n $'\x$i'"; done
But wait, there's more
Bash has a C-style for loop
$ for ((i = 0; i <= 10; i+=2)); do echo -n "$i "; done; echo
0 2 4 6 8 10
Which has the exact same functionality as:
$ for i in $(seq 0 2 10); do echo -n "$i "; done; echo
0 2 4 6 8 10
Bash can handle anything that the expr command can:
$ echo $((5 % 2))
1
$ echo $((5 / 2))
2
$ echo $((5**2))
25
And then some:
$ echo $(( (5**2) < 30 ))
1
Wouldn't it be nice to know you don't have a dependency before you waste time running a script?
Bash has two commands that can do this:
$ command -v [
[
$ echo $?
0
$ type [
[ is a shell builtin
$ echo $?
0
There are differences, but the main one from a UX perspective is that command won't output to STDERR if the command doesn't exist:
$ type blah >/dev/null 2>&1
$ command -v blah >/dev/null
So for a sanity check, something like this works:
if ! command -v printf >/dev/null 2>&1; then
echo "We can not find the bash printf command" >&2
exit 1
fi
Since we'll probably want to die in multiple parts of the script, having a die() function (or similar) probably isn't a bad idea.
If you want an example, (along with a warn() and debug() function) see here
Reading config files
This is quite simple (and has security implications):
source ~/.config/file >/dev/null 2>&1
Sourced files are executable - using them as config data is kind of a hack
$ echo "echo foo" > sourced
$ source ./sourced
foo
Notes:
foo=$(rm -rf /*)
Reading data files
To slurp a file into a variable (while and mapfile can also do similar):
$ echo $(< foo.csv)
foo,bar,baz aaa,bbb,ccc
$ echo "$(< foo.csv)"
foo,bar,baz
aaa,bbb,ccc
Modifying input as it gets read:
$ mapfile -t -c 1 -C 'echo $@' < foo.csv
0 foo,bar,baz
1 aaa,bbb,ccc
$ while read line; do echo $line; done < foo.csv
foo,bar,baz
aaa,bbb,ccc
This is similar to the ||= or //= operator from other languages
This looks ugly (so try to avoid it)
[[ -z foo ]] && foo="something"
This looks slightly cleaner
: "${foo:="something"}"
Grabbing the first column
From a file
$ while IFS="," read -a data; do
echo "${data[0]}"
done < foo.csv
foo
aaa
Or directly from a string:
$ while IFS="," read -a data; do
echo "${data[0]}"
done <<< "aaa,bbb,ccc"
aaa
Bash has 5 variable types: string, integer, array, associative array (aka dictionary or hash), and reference. You normally only care about strings and arrays.
Though associative arrays can be useful when you know you will always run on a newer bash versionAll variables may be defined with the respective switch to declare, or bash will determine the type based on how they are defined
Verbose declaration
$ unset f; declare f=; declare -p f
declare -- f=""
$ unset f; declare -a f=; declare -p f
declare -a f='([0]="")'
$ unset f; declare -A f=; declare -p f
declare -A f='([0]="" )'
Implied declation
$ unset f; f=; declare -p f
declare -- f=""
$ unset f; f=(); declare -p f
declare -a f='()'
Manipulating arrays is really powerful
$ echo "${foo[@]/a/A}"
Aaa bbb ccc
$ echo "${foo[@]/a*/A}"
A bbb ccc
$ echo "${foo[@]/#a/A}"
Aaa bbb ccc
Array manipulation for commands
$ f=(foo bar baz)
$ find -type f -name foo ${f[@]/#/-o -name }
Bash functions are better than most languages (they can be scoped and redefined on the fly - not as powerful as vimscript but pretty good)
An example of a function I have in my shell's rc file
# long dig
ldig () {
dig +trace +nocmd "$@" any +multiline +answer
}
Passing functions to other sessions
$ _f () { whoami; }
$ sudo bash -c "$(declare -f _f) && _f"
root
$ _f () { groups $1; }
$ sudo bash -c "$(declare -f _f) && _f tuser"
tuser : tuser
Passing arrays to functions is possible by reference
$ func () {
echo "zero [${!0}]" \
"one [${!1}]" \
"two [${!2}]" \
"three [${!3}]";
}
$ func foo[@] i
zero [] one [aaa bbb ccc] two [5] three []
Again, don't use subshells (they're expensive)
Don't do this
func () {
echo "something"
}
foo=$(func)
I use RET_<FUNC NAME> as a pattern for return variables or RET_<FUNC NAME>_<THING> if returning multiple variables
declare F_RET=""
f () {
F_RET=""
......
}
And you can redefine functions on the fly
f () {
f {} {
echo "foo"
}
f $@
}
Use set's x option
f () {
set -ex
....
set +ex
}
Note on -e (aka errexit) - it is an old POSIX atandard (since revised). Will exit when $? != 0
$ unset i; i=1; (( i-- )); echo $?; (( i-- )); echo $?
0
1
$ set -e
$ unset i; i=1; (( i-- )); echo $?; (( i-- )); echo $?
0
And the same statement used to pass a function to another environment (subshell, sudo, ssh, etc) is quite useful here too
$ declare -f echo
echo ()
{
command echo $@
}
File descriptor/handle
If a program wants a file handle and you want to give input instead
echo "foo" | cat /dev/fd/0
Or another thing I include in my rc file (to tell ssh to ignore the config file)
alias sshn="ssh -F /dev/null"
alias scpn="scp -F /dev/null"
Shell compatability
Tell which shells some syntax works with
for sh in dash mksh zsh ksh93 posh 'busybox sh' bash; do
$sh -c 'a=b b=2; echo $((a++)) $a $b'
done
Profiling
The reason I say not to use subshells when at all possible
$ time for i in {0..1000}; do unset f; f="$((5+2))"; done
real 0m0.037s
user 0m0.037s
sys 0m0.000s
$ time for i in {0..1000}; do unset f; f="$(expr 5 + 2)"; done
real 0m5.219s
user 0m1.582s
sys 0m3.595s