Category: bash

Bash arrays

Bash arrays are a lot like Bash Associative Arrays, but with numbers as keys.

Here’s a quick reference.

Basics

$ declare -a MYARR  # Create an array
$ MYARR[3]=foo      # Put a value into an array
$ echo ${MYARR[3]}  # Get a value out of an array
foo
$ echo MYARR[3]     # WRONG
MYARR[0]
$ echo $MYARR[3]]   # WRONG
[3]

Creating, adding

$ declare -a MYARR    # Explicitly declare
$ MYARR[3]=foo        # Or this line implicitly makes it an array
$ MYARR[4]=bar        # Can add values one by one

$ declare -a MYARR=(a b c)   # Initialise all at once
$ echo ${MYARR[0]}
a
$ echo ${MYARR[1]}
b
$ echo ${MYARR[2]}
c

$ declare -a MYARR   # Or declare separately
$ MYARR=(a b c)      # Then initialise
$ echo ${MYARR[0]}
a
$ echo ${MYARR[1]}
b
$ echo ${MYARR[2]}
c

$ declare -a MYARR=(a b c)
$ MYARR=("${MYARR[@]}" d)  # Add an element
$ echo ${MYARR[@]}
a b c d
$ declare -a MYARR2=(e f g)
$ MYARR=("${MYARR[@]}" "${MYARR2[@]}")  # Concatenate arrays
$ echo ${MYARR[@]}
a b c d e f g

Keys/Indices

$ declare -a MYARR
$ MYARR[3]=foo
$ echo ${MYARR[0]}  # Unassigned values are empty

$ echo ${MYARR[4]}  # Unassigned values are empty

$ MYARR[seven]=bar     # A text index is treated as 0
$ echo ${MYARR[0]}
bar
$ echo ${MYARR[seven]} # A text index is treated as 0
bar

$ K=3
$ MYARR[$K]=baz      # Variables containing numbers work like numbers
$ echo ${MYARR[$K]}
baz
$ echo ${MYARR[3]}   # Obviously the value is accessible via the actual index
baz
$ K=foo
$ MYARR[$K]=bash     # Variables containing text are treated as 0
$ echo ${MYARR[0]}
bash

Length

$ declare -a MYARR=(a b c)
$ echo ${#MYARR[@]}  # Length of an array
3
$ echo $#MYARR[@]  # WRONG
0MYARR[@]
$ echo ${#MYARR}   # WRONG
1

$ MYARR[7]=x
$ echo ${#MYARR[@]}  # Only existing indices count in the length
4

$ declare -a MYARR=(a bb ccc)
$ echo ${#MYARR[0]}   # Length of an individual element
1
$ echo ${#MYARR[1]}
2
$ echo ${#MYARR[2]}
3

Looping

$ declare -a MYARR=("a 1" b c)
$ # Loop through array values
$ for V in "${MYARR[@]}"; do echo $V; done
a 1
b
c
$ for V in ${MYARR[@]}; do echo $V; done  #WRONG
a
1
b
c

$ echo "${!MYARR[@]}"  # Print all indices - quoted, but quotes removed by echo
0 1 2
$ echo "${MYARR[@]}"   # Print all values - quoted, but quotes removed by echo
a 1 b c

Clearing

$ declare -a MYARR
$ MYARR[3]=x

$ echo ${MYARR[3]}
x
$ unset MYARR
$ declare -a MYARR
$ echo ${MYARR[3]}

Deleting

$ MYARR[2]=foo
$ echo ${MYARR[2]}
foo
$ unset ${MYARR[2]} # WRONG
$ echo ${MYARR[2]}
foo
$ unset MYARR[2]    # To delete from an array, use "unset" with similar syntax to assigning
$ echo ${MYARR[2]}

$ MYARR[3]=quux
$ echo ${MYARR[3]}
quux
$ K=3
$ unset MYARR[$K]   # Can unset using a variable for the key too
$ echo ${MYARR[3]}


$ declare -a MYARR=(a b c d e f)
$ MYARR=("${MYARR[@]:0:3}" "${MYARR[@]:4}")  # Remove element 3, leaving no gap
$ echo ${MYARR[@]}

Cool stuff

$ declare -a MYARR=(a b c d e f g)
$ echo ${MYARR[@]:2:3}              # Extract a sub-array
c d e

$ declare -a MYARR=(a b c d e f g)
$ echo ${MYARR[@]/d/FOO}            # Replace elements that match
a b c FOO e f g

Scope

$ unset MYARR
$ function createmap() { MYARR[5]=bar; }  # Implicit creation puts it in the global scope
$ echo ${MYARR[5]}

$ createmap
$ echo ${MYARR[5]}
bar

$ unset MYARR
$ function createmaplocal() { declare -a MYARR; MYARR[3]=bar; }  # Explicit creation puts it in the local scope
$ echo ${MYARR[3]}

$ createmaplocal
$ echo ${MYARR[3]}

Links

Passing several values through a pipe in bash

I have been fiddling with some git-related shell scripts, and decided to try and follow the same approach as git in their structure. This means using the Unix system where each piece of functionality is a separate script (or executable) that communicates by using command-line arguments, reading from the standard input stream, and writing output to the standard output stream.

This allows each piece of functionality to be written in any programming or scripting language. In git’s case this has allowed initial versions to be written in bash or perl and later optimised versions (sometimes written in C) to be dropped in, piece by piece. It’s an incredibily flexible way of working and can also be very efficient.

Most of my prototyping has been in bash, and I’ve found sometimes I need to write out multiple values from a script and collect them as input in another script.

Writing the output is simple:

#!/bin/bash

# outputter.bash

# Imagine A, B and C have been created by some complex process:
A="foo bar"
B="  bar"
C="baz   "

# At the end of our script we simply write them out on separate lines in a known order
echo "${A}"
echo "${B}"
echo "${C}"

But reading them in somewhere else gave me some trouble until I learned this recipe:

#!/bin/bash

# inputter.bash

# Read in the values one per line:
IFS=$'\n' read A
IFS=$'\n' read B
IFS=$'\n' read C

# Now we can use them.
echo "A='${A}'"
echo "B='${B}'"
echo "C='${C}'"

And now the values transfer succesfully, preserving whitespace:

$ ./outputter.bash | ./inputter.bash 
A='foo bar'
B='  bar'
C='baz   '

The recipe uses bash’s built-in read command to populate the variables, but sets the IFS variable (Internal Field Separator) to a newline, meaning all the whitespace in the line is treated as part of the value to be read. The $'\n' syntax is a literal newline.

Bash associative array examples

Quick reference of things I discovered about how to use associative arrays in bash. Note: bash version 4 only.

Play my Android game Rabbit Escape! Get it on Google Play

Update: see also Bash Arrays.

Basics

$ declare -A MYMAP     # Create an associative array
$ MYMAP[foo]=bar       # Put a value into an associative array
$ echo ${MYMAP[foo]}   # Get a value out of an associative array
bar
$ echo MYMAP[foo]      # WRONG
MYMAP[foo]
$ echo $MYMAP[foo]     # WRONG
[foo]

Creating

$ declare -A MYMAP     # Explicitly declare
$ MYMAP[foo]=bar       # Or this line implicitly makes it an associative array (in global scope, bash 4.2+ only)
$ MYMAP[baz]=quux      # Can add multiple values one by one
$ MYMAP[corge]=grault

$ declare -A MYMAP=( [foo]=bar [baz]=quux [corge]=grault ) # Initialise all at once
$ echo ${MYMAP[foo]}
bar
$ echo ${MYMAP[baz]}
quux

$ declare -A MYMAP                              # Or declare separately
$ MYMAP=( [foo]=bar [baz]=quux [corge]=grault ) # Then initialise
$ echo ${MYMAP[foo]}
bar
$ echo ${MYMAP[baz]}
quux

Variables as keys

$ K=baz
$ MYMAP[$K]=quux       # Use a variable as key to put a value into an associative array
$ echo ${MYMAP[$K]}    # Use a variable as key to extract a value from an associative array
quux
$ echo ${MYMAP[baz]}   # Obviously the value is accessible via the literal key
quux

Quoting keys

$ declare -A MYMAP
$ MYMAP[foo A]="bar B"         # Quoting keys makes no difference
$ MYMAP["corge X"]="grault Y"  # Quoting keys makes no difference
$ echo ${MYMAP["corge X"]}     # You can access by quoting
grault Y
$ echo ${MYMAP[corge X]}       # Or not bother
grault Y
$ echo ${MYMAP[foo A]}
bar B

$ MYMAP['waldo 1']="fred 2"    # Single quotes also make no difference
$ echo ${MYMAP['waldo 1']}
fred 2
$ echo ${MYMAP[waldo 1]}
fred 2

$ K=plugh
$ MYMAP['$K']=xyzzy   # Except single quotes prevent variable expansion, as usual
$ echo ${MYMAP[plugh]}

$ echo ${MYMAP['$K']}
xyzzy

Missing keys

$ MYMAP[foo]=bar
$ echo ${MYMAP[missing]}   # Accessing a missing value gives ""

$ # Testing whether a value is missing from an associative array
$ if [ ${MYMAP[foo]+_} ]; then echo "Found"; else echo "Not found"; fi
Found
$ if [ ${MYMAP[missing]+_} ]; then echo "Found"; else echo "Not found"; fi
Not found

Looping

$ declare -A MYMAP=( [foo a]=bar [baz b]=quux )
$ echo "${!MYMAP[@]}"  # Print all keys - quoted, but quotes removed by echo
foo a baz b

$ # Loop through all keys in an associative array
$ for K in "${!MYMAP[@]}"; do echo $K; done
foo a
baz b

$ for K in ${!MYMAP[@]}; do echo $K; done     # WRONG
foo
a
baz
b

$ # Looping through keys and values in an associative array
$ for K in "${!MYMAP[@]}"; do echo $K --- ${MYMAP[$K]}; done
foo a --- bar
baz b --- quux

$ # Loop through all values in an associative array
$ for V in "${MYMAP[@]}"; do echo $V; done
bar
quux

Clearing

$ declare -A MYMAP
$ MYMAP[foo]=bar
$ echo ${MYMAP[foo]}
bar
$ declare -A MYMAP    # Re-declaring DOES NOT clear an associative array
$ echo ${MYMAP[foo]}
bar
$ unset MYMAP         # You need to unset and re-declare to get a cleared associative array
$ declare -A MYMAP
$ echo ${MYMAP[foo]}

Deleting keys

$ MYMAP[foo]=bar
$ echo ${MYMAP[foo]}
bar
$ unset ${MYMAP[foo]} # WRONG
$ echo ${MYMAP[foo]}
bar
$ unset MYMAP[foo]    # To delete from an associative array, use "unset" with similar syntax to assigning
$                     # BUT see next section if key contains spaces
$ echo ${MYMAP[foo]}

$ MYMAP[baz]=quux
$ echo ${MYMAP[baz]}
quux
$ K=baz
$ unset MYMAP[$K]       # Can unset using a variable for the key too
$                       # BUT see next section if variable may contain spaces - always better to quote
$ echo ${MYMAP[baz]}

Deleting keys containing spaces

$ declare -A MYMAP
$ MYMAP[foo Z]=bar
$ echo ${MYMAP[foo Z]}
bar
$ unset MYMAP[foo Z]     # WRONG
bash: unset: `MYMAP[foo': not a valid identifier
bash: unset: `Z]': not a valid identifier
$ unset MYMAP["foo Z"]    # You must quote keys containing spaces when you unset in an associative array
$ echo ${MYMAP[foo Z]}

$ MYMAP[foo Z]=bar
$ unset MYMAP['foo Z']    # Single quotes work too
$ echo ${MYMAP[foo Z]}

$ MYMAP[baz A]=quux
$ echo ${MYMAP[baz A]}
quux
$ K="baz A"
$ unset MYMAP[$K]         # WRONG
bash: unset: `MYMAP[baz': not a valid identifier
bash: unset: `A]': not a valid identifier
$ unset MYMAP["$K"]       # You must quote variables whose values may contain spaces when you unset in an associative array
$ echo ${MYMAP[baz A]}

$ MYMAP[baz A]=quux
$ unset MYMAP['$K']       # Curiously, single quotes work here too, although I'd suggest avoiding them
$ echo ${MYMAP[baz A]}

Length

$ declare -A MYMAP=( [foo a]=bar [baz b]=quux )
$ echo ${#MYMAP[@]}  # Number of keys in an associative array
2
$ echo $#MYMAP[@]  # WRONG
0MYMAP[@]
$ echo ${#MYMAP}   # WRONG
0

Numeric indexing

$ declare -A MYMAP=( [foo a]=bar [baz b]=quux )
$ KEYS=("${!MYMAP[@]}")      # Make a normal array containing all the keys in the associative array
$ echo ${KEYS[0]}            # Find a key via an index
foo a
$ echo ${MYMAP[${KEYS[0]}]}  # Find a value via an index
bar

$ # Loop through using an index
$ for (( I=0; $I < ${#MYMAP[@]}; I+=1 )); do KEY=${KEYS[$I]};  echo $KEY --- ${MYMAP[$KEY]}; done
foo a --- bar
baz b --- quux

Scope

$ unset MYMAP
$ function createmap() { MYMAP[foo]=bar; }  # Implicit creation puts it in the global scope
$ echo ${MYMAP[foo]}

$ createmap
$ echo ${MYMAP[foo]}
bar

$ unset MYMAP
$ function createmaplocal() { declare -A MYMAP; MYMAP[foo]=bar; }  # Explicit creation puts it in the local scope
$ echo ${MYMAP[foo]}

$ createmaplocal
$ echo ${MYMAP[foo]}

Checking Bash version

$ bash --version   # Must be at least version 4 to have associative arrays
GNU bash, version 4.2.24(1)-release (x86_64-pc-linux-gnu)
...

Links