How to Remove All Duplicate Values From a Slice in Golang

Removing duplicate items from a Golang slice ensures that your data remains accurate and consistent. These identical values can lead to misleading or incorrect results in computations, analysis, or data processing.

Today, you will learn how easy it is to remove all the duplicate values from a slice in Golang. This method works on a slice of any type.

Removing Duplicate Value From Golang Slice Using Map

To remove duplicate values from a Golang slice, one effective method is by using maps. Golang map stores data as key-value pairs. It can track the unique elements within the slice and generate a new slice only with these distinct values.

Let's create a function called removeDuplicates that takes in a slice of strings as input and returns a new slice that contains only the unique values.

          func removeDuplicates(slice []string) []string {
	uniqueMap := make(map[string]bool)

	uniqueSlice := []string{}

	// Iterate over the original slice
	for _, value := range slice {
		_, ok := uniqueMap[value]

		if !ok {
			uniqueMap[value] = true
			uniqueSlice = append(uniqueSlice, value)
		}
	}

	return uniqueSlice
}
        

We create an empty map called uniqueMap using the make() function. This map will help us to check if an item is unique or not.

Next, we have an empty slice called uniqueSlice. This slice will eventually hold the unique values extracted from the original slice.

Now iterate over the original slice using the range syntax. You will have access to each element of the slice in this value variable inside this loop.

Also Read: Deep Dive into Variable Scopes in Golang: Learn with Examples

You need to access the uniqueMap with the value as a key. It will return 2 things: the actual value of that key and a boolean. In this example, we only want the boolean.

If the value key already exists in the uniqueMap, the ok will be true. It will indicate that this value element is a duplicate in the slice. If the value is not present in the uniqueMap (i.e., ok is false), it means it is a unique value.

In this case, you will add the value to the uniqueSlice by using the append function and simultaneously, mark it as seen in the uniqueMap by setting its value to true.

Once the loop completes, this function returns the uniqueSlice, which contains only the unique values in the original order.

          package main

import "fmt"

func removeDuplicates(slice []string) []string {
	uniqueMap := make(map[string]bool)

	uniqueSlice := []string{}

	// Iterate over the original slice
	for _, value := range slice {
		_, ok := uniqueMap[value]

		if !ok {
			uniqueMap[value] = true
			uniqueSlice = append(uniqueSlice, value)
		}
	}

	return uniqueSlice
}

func main() {
	slice := []string{"JavaScript", "Golang", "Python", "PHP", "JavaScript", "C++", "Python"}

	// Remove duplicates from the slice
	uniqueSlice := removeDuplicates(slice)

	fmt.Println(uniqueSlice)
}
        

Here I have a slice of strings that contains some identical values. If I pass this slice to the removeDuplicates() function, it will return a slice of string only with the unique value.

By utilizing a map to track unique values, this approach ensures that duplicates are efficiently filtered out, resulting in a new slice that exclusively holds the unique elements.

Also Read: Functions VS Methods in Golang: Understanding the Differences