Golang remove duplicates from slice with Examples


Written By - Mohamed Allam
Advertisement

Go slices, is a data structure, a list of values in a memory, Go arrays are very powerful, they are the building block of Slices have and are extremely powerful. In a slice, we need to declare the size, at compile time, so we can hard code it or we can defer to the compiler. However that size will be fixed along the way, and cannot be changed.

The issue we might come across. is duplicate items in a slice. So let’s figure out how to deal with that.

 

Retrieve unique elements from slice (remove duplicates)

Given we have this slice of integers, where we know there is one element that has not been duplicated, to retrieve that element in the most efficient way, we can use XOR

package main

import (
	"fmt"
)

func uniqueeNonDuplicate(array []int) int {
	unique := 0
	for i := 0; i < len(array); i++ {
		unique ^= array[i]

	}
	return unique
}

func main() {
	// 5 is the unique element here 
	array := []int{8, 7, 5, 0, 0, 1, 2, 2, 8, 3, 4, 3, 4, 1, 6, 6, 7}

	fmt.Println(uniqueeNonDuplicate(array))
}

To print out

5

 

Count duplicate elements in a slice

In this approach we will count the number of duplicate items present in a slice in golang. To do so, we will first retrieve the greatest element in the slice, 5 for our example, and create a slice, with. 6 elements, then we can just increase the count for each element

package main

import (
	"fmt"
)

func findGreat(array []int) int {
	greatest := 0
	for i := 0; i < len(array); i++ {
		if array[i] > greatest {
			greatest = array[i]
		}
	}
	return greatest + 1
}

func findDup(array []int) []int {
	dup := make([]int, findGreat(array))

	for i := 0; i < len(array); i++ {
		dup[array[i]] += 1
	}
	return dup
}
func main() {
	array := []int{8, 7, 5, 0, 0, 1, 2, 2, 8, 3, 4, 3, 4, 1, 6, 6, 7}
	fmt.Println(findDup(array))

	for i, v := range findDup(array) {
		fmt.Printf("value %v, repetitions %v\\n", i, v)
	}
}

To print out

[2 2 2 2 2 1 2 2 2]
value 0, repetitions 2
value 1, repetitions 2
value 2, repetitions 2
value 3, repetitions 2
value 4, repetitions 2
value 5, repetitions 1
value 6, repetitions 2
value 7, repetitions 2
value 8, repetitions 2

This can be messy, as you might have guessed, if we have a value, lets say 23, we would have zero values for all elements between 8, and 23, therefor we can reach out to maps.

 

Remove duplicate items and keep one unique item inside the slice

If we have a slice of integers, where we want to remove duplicate ones, and append them to a new array keeping atleast one unique element from each duplicate item. Let's check this example:

Advertisement
package main

import (
	"fmt"
)

func keepOne(array []int) []int {
	keepOne := make([]int, 0)
	for i := 0; i < len(array); i++ {
		if elementExists(keepOne, array[i]) == false {
			keepOne = append(keepOne, array[i])
		}
	}
	return keepOne
}

func elementExists(haystack []int, needle int) bool {
	for _, v := range haystack {
		if v == needle {
			return true
		}
	}
	return false
}

func main() {
	array := []int{8, 7, 5, 0, 0, 1, 2, 2, 8, 3, 4, 3, 4, 1, 6, 6, 7}
	fmt.Println(array)
	fmt.Println(keepOne(array))
}

To print out

[8 7 5 0 0 1 2 2 8 3 4 3 4 1 6 6 7]
[8 7 5 0 1 2 3 4 6]

 

Count duplicate elements and store them in a map

In this example we will go through all the slice elements and then store the individual elements along with their count inside a new map.

package main

import "fmt"

func findDupMap(array []int) map[int]int {

	dup := make(map[int]int)

	for i := 0; i < len(array); i++ {
		dup[array[i]] += 1
	}
	return dup
}

func main() {
	array := []int{23, 8, 7, 5, 0, 0, 1, 2, 2, 8, 3, 4, 3, 4, 1, 6, 6, 7}

	fmt.Println(findDupMap(array))
	fmt.Printf("Element %v repeated %v\\n", 23, findDupMap(array)[23])

	fmt.Println()
	for i, v := range findDupMap(array) {
		fmt.Printf("Element %v repeated %v\\n", i, v)
	}

}

To print out

map[0:2 1:2 2:2 3:2 4:2 5:1 6:2 7:2 8:2 23:1]
Element 23 repeated 1

Element 8 repeated 2
Element 0 repeated 2
Element 2 repeated 2
Element 3 repeated 2
Element 4 repeated 2
Element 6 repeated 2
Element 23 repeated 1
Element 7 repeated 2
Element 5 repeated 1
Element 1 repeated 2

And this gives us granular control over what we can do, to every item, or repetition.

 

 Conclusion

If you just need to check duplication, XOR can solve that. Slice can do be useful as a count, or return a slice with no duplication. If you want maximum flexibility use a map, to retrieve, order, sort or do any manipulation. In this tutorial we covered

  • remove duplicates in slice golang
  • find unique elements in array golang
  • golang count occurrences in slice
  • find duplicates in array golang
  • Remove duplcate elements and create a map

 

Further Reading

How to remove duplicates strings or int from Slice in Go

 

Categories GO

Didn't find what you were looking for? Perform a quick search across GoLinuxCloud

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

Buy GoLinuxCloud a Coffee

For any other feedbacks or questions you can either use the comments section or contact me form.

Thank You for your support!!

Leave a Comment