Bash Remove Element from Array Smoothly [11 Methods]

In bash scripting, handling arrays and their elements is a crucial skill. A common task is to remove specific elements from an array to meet various needs. Bash offers multiple methods to do this, like using loops, built-in commands, and external utilities. Keywords such as bash remove element from array, bash remove from array, and bash remove item from array are central to these operations. They help in modifying the array by removing specific values or items. Mastering these operations and methods is essential. They make bash scripting more flexible and dynamic, allowing for better data management and manipulation within scripts. These techniques are vital for those who want to manage arrays efficiently in bash.

 

Different methods to remove element from array in Bash

1. Using Loops: Iterate through each element of the first array, checking if it exists in the second array, and then remove it if it does.
2. Using grep Command: Use the grep command with the -v option to exclude the elements of the first array from the second array.
3. Using awk command: Employ the awk command to compare the elements of two arrays and exclude the common elements.
4. Using comm Command: The comm command can be used to compare two files (or the output of commands) line by line, making it possible to remove elements of one array present in another.
5. Using diff Command: The diff command can help in identifying the differences between two arrays and excluding the common elements.
6. Using sort and uniq Commands: Sort both arrays and then use uniq to remove duplicate elements, achieving the exclusion of common elements.
7. Using Array Difference in Bash: Utilize native bash operations to find the difference between arrays and create a new array with unique elements.
8. Using sed Command: Use the sed command to manipulate the text output of arrays and remove common elements.
9. Using Associative Arrays: Convert one array into an associative array (hash map) and then exclude its elements from the other array.
10. Using External Tools like jq: Tools like jq can be used for more complex operations, like when dealing with arrays of JSON objects, to exclude elements from one array in another.
11. Using Parameter Expansion: Bash parameter expansion can be utilized for string manipulation to remove elements from one array present in another.

 

1. Using Loops

In this method, bash remove element from array is accomplished by iterating through each element in the target array. If an element matches an item we intend to remove, it is skipped, effectively removing it from the array.

array1=("a" "b" "c" "d")
element_to_remove="b"

for item in "${array1[@]}"; do
    if [[ $item != $element_to_remove ]]; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• A loop is utilized to go through each element (item) in array1.
• The element intended for removal (element_to_remove) is compared with each array element.
• If an element doesn’t match the element_to_remove, it’s added to a new array (new_array), effectively performing the "bash remove item from array" operation.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

for element in "${array_to_remove[@]}"; do
    array1=("${array1[@]/$element}")
done

echo "${array1[@]}"

• Loops through each element in the array_to_remove.
• For each element in array_to_remove, it replaces occurrences in array1 with an empty string.
• Handles non-existing elements by leaving the array as is, making no replacements.

 

2. Using grep Command

The grep command allows us to "bash remove from array" by excluding specific elements. It filters out the undesired elements, returning a refined array.

array1=("a" "b" "c" "d")
element_to_remove="b"
new_array=($(for item in "${array1[@]}"; do echo "$item"; done | grep -v $element_to_remove))

echo "${new_array[@]}"

• for item in "${array1[@]}"; do echo "$item"; done - This part of the command loops through each element in array1, printing each one on a new line.
• grep -v $element_to_remove - This excludes lines matching the element_to_remove.
• The remaining lines (array elements) are saved in the new_array.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

new_array=()

for item in "${array1[@]}"; do
    if ! echo "${array_to_remove[@]}" | grep -q -w "$item"; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• Prints each element of array1 on a new line.
• grep excludes lines that match any element in array_to_remove.
• Non-matching elements (non-existing in array_to_remove) are retained, resulting in the modified array1.

 

3. Using awk Command

awk is a potent tool used to "bash remove item from array" by matching and excluding specific elements during the processing of array data.

array1=("a" "b" "c" "d")
element_to_remove="b"
new_array=($(for item in "${array1[@]}"; do echo "$item"; done | awk -v var="$element_to_remove" '$0 != var'))

echo "${new_array[@]}"

• The for loop iterates over each element in array1, printing each one on a new line.
• This output is piped into awk, where it is compared to element_to_remove.
• awk filters out the matching element, and the remaining elements are saved to new_array.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

new_array=()

for item in "${array1[@]}"; do
    found=0
    for remove in "${array_to_remove[@]}"; do
        if [[ $(echo "$item" | awk -v var="$remove" '$0 ~ var {print $0}') ]]; then
            found=1
            break
        fi
    done
    if [[ $found -eq 0 ]]; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• Two nested loops are used. The outer loop iterates over each element in array1, and the inner loop iterates over each element in array_to_remove.
• In the inner loop, awk compares each element from array1 with each element in array_to_remove.
• If an element from array1 is found in array_to_remove, a flag (found) is set.
• After the inner loop finishes, if the flag is not set, the element from array1 is added to new_array.
• Finally, new_array contains all elements from array1 that are not present in array_to_remove.

 

4. Using comm Command

This method involves using the comm command, which is typically used to compare sorted files, for the "bash remove element from array" operation by comparing and excluding array elements.

array1=("a" "b" "c" "d")
array_to_remove=("b")
new_array=($(comm -23 <(echo "${array1[*]}" | tr ' ' '\n' | sort) <(echo "${array_to_remove[*]}" | tr ' ' '\n' | sort)))

echo "${new_array[@]}"

• Both arrays are sorted and provided as input to the comm command.
• The -23 option ensures that only unique elements of the first array are shown, effectively removing common elements, thereby achieving the "bash remove item from array" operation.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

new_array=($(comm -23 <(echo "${array1[@]}" | tr ' ' '\n' | sort) <(echo "${array_to_remove[@]}" | tr ' ' '\n' | sort)))

echo "${new_array[@]}"

• Elements from each array are sorted and displayed line by line.
• comm identifies common lines and lines exclusive to each array.
• Only lines exclusive to array1 are kept, representing the array after the removal of the specified elements.

 

5. Using diff Command

This method employs the diff command to identify and "bash remove element from array" by isolating differences between two arrays and outputting the unique elements.

array1=("a" "b" "c" "d")
array_to_remove=("b")
new_array=($(diff <(echo "${array1[*]}" | tr ' ' '\n') <(echo "${array_to_remove[*]}" | tr ' ' '\n') | grep '<' | cut -d' ' -f2))

echo "${new_array[@]}"

• The diff command compares the contents of two arrays, line by line.
• Only the differences, prefixed with <, are extracted, then processed to remove the prefix, accomplishing the "bash remove from array" task.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

new_array=($(diff <(echo "${array1[*]}" | tr ' ' '\n') <(echo "${array_to_remove[*]}" | tr ' ' '\n') | grep '<' | cut -d' ' -f2))

echo "${new_array[@]}"

• Elements of each array are displayed as separate lines in temporary files.
• diff identifies differences between the two sets of lines.
• Only the lines exclusive to array1 are retained, representing the modified array.

 

6. Using sort and uniq Commands

By using sort in conjunction with uniq, we can "bash remove item from array" by identifying and excluding duplicate or specified items from the array.

array1=("a" "b" "c" "d" "b")
array_to_remove=("b")
new_array=($(echo "${array1[@]}" "${array_to_remove[@]}" | tr ' ' '\n' | sort | uniq -u))

echo "${new_array[@]}"

• Both arrays are combined, and the elements are sorted.
• The uniq -u command is used to filter out the non-unique elements, performing the "bash remove element from array" action.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

new_array=($(echo "${array1[@]}" "${array_to_remove[@]}" "${array_to_remove[@]}" | tr ' ' '\n' | sort | uniq -u))

echo "${new_array[@]}"

• Elements from all arrays are combined, each being echoed on a new line.
• They are sorted and duplicates (elements to be removed) are identified.
• uniq -u prints only the unique elements, effectively removing the duplicates.

 

7. Using Associative Arrays

Associative arrays or hash maps in bash are instrumental to "bash remove item from array." They assist in managing and excluding specific elements efficiently.

declare -A array_to_remove=( ["b"]=1 )
array1=("a" "b" "c" "d")

for item in "${array1[@]}"; do
    if [[ -z "${array_to_remove[$item]}" ]]; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• An associative array is created with elements to be removed.
• A loop checks each element of array1 against the associative array, adding only the ones not found in the associative array, thus executing the "bash remove from array" process.

declare -A array_to_remove=( ["b"]=1 ["e"]=1 )
array1=("a" "b" "c" "d")

for item in "${array1[@]}"; do
    [[ ! ${array_to_remove[$item]} ]] && new_array+=("$item")
done

echo "${new_array[@]}"

• An associative array is created for easier lookup.
• Each element of array1 is checked against the keys of the associative array, and non-matching elements are added to the new array.

 

8. Using sed Command

The sed command is utilized in this method to perform text manipulation, aiding in the "bash remove element from array" process by excluding specified patterns or elements.

array1=("a" "b" "c" "d")
element_to_remove="b"
new_array=($(echo "${array1[@]}" | sed "s/$element_to_remove//"))

echo "${new_array[@]}"

• Elements of array1 are piped into the sed command.
• The sed command replaces the specified element with an empty string, effectively performing the "bash remove item from array" operation. Note that this might leave an empty array element if the removed element is not at the end or beginning.

array1=("a" "b" "c" "d")
array_to_remove=("b" "c")

new_array=("${array1[@]}")

for element_to_remove in "${array_to_remove[@]}"; do
    new_array=($(for item in "${new_array[@]}"; do echo "$item"; done | sed "/$element_to_remove/d"))
done

# Printing the new array
echo "${new_array[@]}"

• This script also begins with two arrays: array1 and array_to_remove.
• new_array is initialized with all elements from array1.
• For each element in array_to_remove, a loop iterates over new_array and uses sed to remove the current element if it is found.
• After processing all elements in array_to_remove, new_array will only contain elements from array1 that were not in array_to_remove.

 

9. Using External Tools like jq

External tools such as jq can be quite powerful for manipulating arrays, especially when dealing with JSON data structures, allowing us to "bash remove element from array" efficiently.

array1=('{"name":"a"}' '{"name":"b"}' '{"name":"c"}' '{"name":"d"}')
element_to_remove="b"

new_array=()
for item in "${array1[@]}"; do
    if [[ $(echo "$item" | jq -r '.name') != "$element_to_remove" ]]; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• Each item in the array array1 is a string that represents a JSON object.
• In the for loop, each item in array1 is processed.
• echo "$item" | jq -r '.name' extracts the value of the "name" property from each JSON object.
• If the value of the "name" property does not equal element_to_remove, the item is added to the new_array.
• This method effectively removes the specified element from the array, filtering out the JSON object with the "name" property value equal to "b".

array1=('{"name":"a"}' '{"name":"b"}' '{"name":"c"}')
array_to_remove=('{"name":"b"}' '{"name":"d"}')

# Create an empty array to store the new filtered items
new_array=()

# Loop through each element in array1
for item1 in "${array1[@]}"; do
    match_found=false
    for item2 in "${array_to_remove[@]}"; do
        # Compare the 'name' value in each JSON object
        if [[ $(echo "$item1" | jq -r '.name') == $(echo "$item2" | jq -r '.name') ]]; then
            match_found=true
            break
        fi
    done
    if [[ $match_found == false ]]; then
        new_array+=("$item1")
    fi
done

# Print the new array
echo "${new_array[@]}"

• This script contains two loops to iterate over elements of array1 and array_to_remove.
• Each element (JSON-like string) in both arrays is processed with jq to extract the value of the name key.
• If the name value of an element in array1 matches any name value in array_to_remove, it is skipped.
• If no match is found, the element from array1 is added to new_array.
• new_array will finally contain elements from array1 that didn’t match any element in array_to_remove.

 

10. Using Array Difference

This method focuses on finding the difference between two arrays, which inherently performs the "bash remove item from array" operation, leaving us with unique elements.

array1=("a" "b" "c" "d")
array_to_remove=("b")
new_array=($(echo "${array1[@]}" "${array_to_remove[@]}" | tr ' ' '\n' | sort | uniq -u))

echo "${new_array[@]}"

• The elements of both arrays are combined, sorted, and duplicates are removed using uniq -u.
• This leaves us with elements that are unique to array1, achieving the "bash remove element from array" task.

array1=("a" "b" "c" "d")
array_to_remove=("b" "e")

# Combine, sort, and get the unique elements that have duplicates
duplicates=($(echo "${array1[@]}" "${array_to_remove[@]}" | tr ' ' '\n' | sort | uniq -d))

# Get elements that are in array1 but not in the duplicates list
new_array=()
for item in "${array1[@]}"; do
    if ! [[ " ${duplicates[@]} " =~ " ${item} " ]]; then
        new_array+=("$item")
    fi
done

echo "${new_array[@]}"

• This script first identifies duplicate elements between array1 and array_to_remove by using uniq -d.
• Then, it creates a new array by adding elements from array1 that are not found in the list of duplicates.
• So, new_array will contain elements that are unique to array1. Elements from array_to_remove that are not in array1 will not be included in new_array.

 

11. Using Parameter Expansion

Parameter expansion in bash is a powerful feature that allows string manipulation directly, facilitating the "bash remove item from array" process by modifying array elements based on specified patterns.

array1=("a" "b" "c" "b")
element_to_remove="b"
new_array=("${array1[@]/$element_to_remove/}")

echo "${new_array[@]}"

• The parameter expansion ${array1[@]/$element_to_remove/} replaces occurrences of the element to remove with an empty string.
• It modifies the array in place, executing the "bash remove from array" action. Note that this might leave an empty array element if the removed element is not at the end or beginning.

array1=("a" "b" "c" "d")
array_to_remove=("b" "c")

new_array=("${array1[@]}")

for element_to_remove in "${array_to_remove[@]}"; do
    new_array=("${new_array[@]/$element_to_remove}")
done

# Printing the new array
echo "${new_array[@]}"

• The code starts with two arrays: array1 which contains the elements we start with, and array_to_remove which contains the elements we want to remove from array1.
• A new array (new_array) is created, initializing it with all elements from array1.
• A loop iterates over each element in array_to_remove, and during each iteration, the current element gets removed from new_array using the pattern substitution feature.
• After all elements in array_to_remove have been processed, the new_array will have all elements from array1 except those that were matched in array_to_remove.

 

Summary

Manipulating arrays is a common necessity in bash scripting, and one frequent operation is the removal of specific elements from an array. Various methods can be utilized to accomplish this task, ensuring flexibility and efficiency according to the specific needs of a script.

Among the notable methods are using loops to iterate over array elements, applying built-in bash commands like grep and awk for pattern matching, and leveraging external utilities such as jq for handling more complex data structures like JSON. Additionally, native bash functionalities such as parameter expansion and process substitution offer powerful tools for array element removal. Uniqueness and difference between arrays can also be managed through the sort and uniq commands, facilitating the identification and exclusion of specific items.

Each method carries its unique advantages and applicability, allowing for the handling of different data types and structures within arrays, providing script developers with a broad toolkit for effective data manipulation and processing in bash.

For a deeper understanding and further exploration of array manipulation in bash, referring to the official GNU Bash documentation is highly recommended: GNU Bash Documentation