Pitfalls of using slice in golang
All memory addresses mentioned in this article are the results generated by the author’s computer at that time, and the actual values will vary depending on each individual’s computer environment. Readers should focus their attention on the logic and essence that the article and the code itself aim to convey.
Before discussing the core of the problem, let’s first talk about pointers in Go.
Go Language Pointers
Go also has pointers, similar to C in concept and syntax, but pointer arithmetic is generally not allowed. For example:
a := 5 // a is an integer with a value of 5
b := &a // b is a pointer to a
c := [3]int{1, 2, 3} // c is an array of size 3
d := &c // d is a pointer to c
Printing the values of a, b, c, and d would yield:
a = 5
&a = 0x20818c538
b = 0x20818c538
*b = 5
c = [1 2 3]
d = 0x2081a0220
&c = 0x2081a0220
*d = [1 2 3]
Nothing special here, of course. You can also change the value of a through the pointer b:
*b = 3 // a = 3, *b = 3
What if you want to change the value of the second element in array c through pointer d? In C, you could do:
*(d+1) = 5; // c = [1 5 3]
But in Go, the compiler will throw you this error:
invalid operation: d + 1 (mismatched types *[3]int and int)
The Go compiler says you can’t add variables of two different types. Notice the compiler considers d’s type as *[3]int, which includes the array size 3, hence arrays of different sizes in Go are also different types. Okay, what if you take a detour, move the pointer forward (d++) and then change the value? The compiler objects again:
invalid operation: d++ (non-numeric type *[3]int)
This is what was meant by Go not allowing pointer arithmetic in general. Oh, then how can I modify the values in array c through pointer d? It’s simple, just operate on pointer d the same way you access a regular array:
d[1] = 5 // c = [1 5 3]
Now, let’s look at an example of a structure pointer:
type student struct {
name string
id int
}
tom := student{name: "Tom", id: 3} // tom is a student structure
mary := &student{name: "Mary", id: 5} // mary is a pointer to a student structure
Print out the students tom and mary:
tom = {Tom 3}
mary = &{Mary 5}
From the & symbol, it’s clear mary is indeed a pointer to a student structure. However, to reference structure members, both the instance variable and the pointer variable use the . operator, as there is no member access operator -> like in C:
println(tom.name) // Tom
println(mary.name) // Mary
That’s about it for pointers. Now, let’s briefly talk about functions in Go.
A Brief Discussion on Go Functions
This introduction to Go functions aims to connect the content to the subsequent sections. Go functions vary more than those in C, and a dedicated article might be written later. For now, let’s look at a few simple examples. First, declare a simple function with no input or output values, which prints a = 5 when called:
func printVar() {
println("a = 5")
}
Then add an input parameter to the function to print the value of a specified variable:
func printVar(a int) {
println(a)
}
Of course, you can also pass in a pointer, similar to C:
func printVar(a *int) {
println(*a) // Prints out 5
*a += 3 // Adds 3 to the value pointed to by the pointer
}
a := 5
printVar(&a)
println(a) // Prints out 8
You can also add a return value to the function to get the modified value:
func printVar(a int) int {
println(a) // Prints out 5
a += 3 // This a is a copy of the original a, so it's actually a' += 3
return a // Returns a'
}
a := 5
a = printVar(a) // Assigns the returned value of printVar() to a
println(a) // Prints out 8
Value copying is just one way functions evaluate expressions, refer to evaluation strategies for Call by value. Next, let’s look at an example of structure passing in functions, declare a clearId() function, its purpose is to receive a pointer to a student structure and set the student’s id field to 0:
type student struct {
name string
id int
}
func clearId(s *student) {
s.id = 0
}
tom := student{name: "Tom", id: 3} // tom is a student structure
mary := &student{name: "Mary", id: 5} // mary is a pointer to a student structure
clearId(&tom) // {Tom 0}
clearId(mary) // &{Mary 0}
Both functions receiving pointers can modify the passed-in values because the parameters are pointers, which is intuitive, so no special explanation is needed, and the results match predictions. That’s the preparation work done; now, let’s get into the main topic.
Utilization of slice in Go Functions
First, look at the following code, slicePlusOne() function accepts a []int type slice, then adds 1 to each element in the slice:
func slicePlusOne(a []int) {
for i := 0; i < len(a); i++ {
a[i]+=1
}
}
a := []int{0, 1, 2, 3, 4}
fmt.Println("Before slicePlusOne() a =", a)
slicePlusOne(a)
fmt.Println(" After slicePlusOne() a =", a)
The result of executing this code is:
Before slicePlusOne() a = [0 1 2 3 4]
After slicePlusOne() a = [1 2 3 4 5]
Good, as expected. Although the function’s parameter doesn’t indicate a is a pointer, the execution result successfully changed the value of slice a, thus naturally, the base type of slice is a pointer. Since it’s a pointer, shouldn’t changing the direction of the pointer change the values inside the slice? Let’s look at another example, sliceChange() function accepts a []int type slice, then sets this slice to the global variable other_slice:
var other_slice = []int{5, 6, 7, 8, 9}
func sliceChange(a []int) {
fmt.Println("Before resetting a =", a)
a = other_slice
fmt.Println(" After resetting a =", a)
}
a := []int{0, 1, 2, 3, 4}
fmt.Println("Before sliceChange() a =", a)
sliceChange(a)
fmt.Println(" After sliceChange() a =", a)
The result of executing this code is:
Before sliceChange() a = [0 1 2 3 4]
Before resetting a = [0 1 2 3 4]
After resetting a = [5 6 7 8 9]
After sliceChange() a = [0 1 2 3 4] // After the function, slice a was not changed
The contents of the slice were not changed! What happened? Let’s modify the program to print out the pointer content and see:
var other_slice = []int{5, 6, 7, 8, 9}
func sliceChange(a []int) {
fmt.Printf("Before resetting a, &a = %v, %p\n", a, &a)
a = other_slice
fmt.Printf("After resetting a, &a = %v, %p\n", a, &a)
}
a := []int{0, 1, 2, 3, 4}
fmt.Printf("Before sliceChange() a, &a = %v, %p\n", a, &a)
sliceChange(a)
fmt.Printf(" After sliceChange() a, &a = %v, %p\n", a, &a)
The result of executing this code is:
Before sliceChange() a, &a = [0 1 2 3 4], pointer address
Before resetting a, &a = [0 1 2 3 4], new pointer address // The pointer address changes upon entering the function
After resetting a, &a = [5 6 7 8 9], new pointer address
After sliceChange() a, &a = [0 1 2 3 4], original pointer address
It’s found that when the slice is passed in, the pointer address is already different! So how did the earlier example, slicePlusOne(), manage to successfully modify? Add print messages inside slicePlusOne() to see the result:
func slicePlusOne(a []int) {
fmt.Printf("Before resetting a, &a = %v, %p\n", a, &a)
for i := 0; i < len(a); i++ {
a[i] += 1
}
fmt.Printf(" After resetting a, &a = %v, %p\n", a, &a)
}
a := []int{0, 1, 2, 3, 4}
fmt.Printf("Before slicePlusOne() a, &a = %v, %p\n", a, &a)
slicePlusOne(a)
fmt.Printf(" After slicePlusOne() a, &a = %v, %p\n", a, &a)
The result of executing this code is:
Before slicePlusOne() a, &a = [0 1 2 3 4], original pointer address
Before resetting a, &a = [0 1 2 3 4], new pointer address // The pointer address changes upon entering the function
After resetting a, &a = [1 2 3 4 5], new pointer address
After slicePlusOne() a, &a = [1 2 3 4 5], original pointer address
Just like the previous example, the pointer address is already different when the slice is passed in! So why did slicePlusOne() manage to modify the values inside the slice, but sliceChange() did not? Here, we need to dive deeper into what a slice type really is in Go.
Go Language Slices
Simply put, a slice is an array without a fixed length. However, at the language implementation level, the slice type is actually a composite type consisting of three parts, as shown in the Golang blog:
- ptr points to the first element in memory that the slice refers to
- len indicates the length of the slice
- cap indicates the capacity of the slice
A slice with both length and capacity of 5, and its relationship with the memory it points to, is illustrated like this in the Golang blog:
Using the following example to illustrate the relationship between a slice and memory:
a := []int{0, 1} // len(a) = 2, cap(a) = 2
b := []int{3, 4} // len(b) = 2, cap(b) = 2
fmt.Printf("Initially a, &a, &a[0], &a[1] = %v, %p, %p, %p\n", a, &a, &a[0], &a[1])
fmt.Printf("Initially b, &b, &b[0], &b[1] = %v, %p, %p, %p\n", b, &b, &b[0], &b[1])
b = a
fmt.Printf("After setting b, &b, &b[0], &b[1] = %v, %p, %p, %p\n", b, &b, &b[0], &b[1])
The result of running this program is:
Initially a, &a, &a[0], &a[1] = [0 1], 0x20819e020, 0x20818a220, 0x20818a228
Initially b, &b, &b[0], &b[1] = [3 4], 0x20819e040, 0x20818a230, 0x20818a238
// After b = a, the address of b[0] = address of a[0]; the address of b[1] = address of a[1]
After setting b, &b, &b[0], &b[1] = [0 1], 0x20819e040, 0x20818a220, 0x20818a228
The address 0x20819e040 pointing to slice b, which stores the slice b structure in memory and is referred to when operating slice b in the program, hasn’t changed. However, the address inside ptr that it points to has changed, now pointing to the memory address inside the slice a structure. Representing this segment in a diagram looks like this:
Using pointers instead of copying memory increases program execution efficiency. From this example, it’s clear that assigning a slice in Go only swaps the memory address pointed to by the internal ptr of the slice structure. So, although slices look and operate like pointers, they are actually a composite structure type that includes a pointer address!
Now, let’s examine another phenomenon when operating slices to deepen our understanding of this type. Previous articles mentioned how to extract a part of a slice into a new slice. The example discussed here is extracting a part of an array into a new slice, with the operation method being exactly the same. Please see the following code:
// Declare an array of length 6
array := [6]int{0, 1, 2, 3, 4, 5}
// Declare a slice equal to all elements within the range 1 <= {x} < 3 in the array
slice := array[1:3]
This operation is clear, the content of the slice will be equal to elements array[1] and array[2]. Print out the slice to verify:
fmt.Println("len(slice) =", len(slice), ", slice =", slice)
// len(slice) = 2 , slice = [1 2]
Everything seems fine, almost as expected. Now, let’s perform another operation on this slice:
// Originally, slice = [1 2]
slice = slice[2:5]
// New slice = ?
How is this operation of extracting a sub-slice of 3 elements, from indices 2 to 5, from a slice that currently has only 2 elements and indices from 0 to 1 possible? Let’s see the result:
fmt.Println("len(slice) =", len(slice), ", slice =", slice)
// len(slice) = 3 , slice = [3 4 5]
Surprisingly, it successfully extracts a new slice with elements completely unrelated to the original slice! How did this happen? How could a slice of length 2 manage to extract 3 elements? Is there something beyond the indices of this slice? Let’s try accessing values beyond the slice’s index to see the result:
fmt.Println("slice[2] =", slice[2])
This results in a runtime error message that crashes the program:
panic: runtime error: index out of range
It seems this slice only has a length of 2, with contents 1 and 2. Up to this point, the information we have isn’t enough for further judgment, so what we need to do is gather more in-depth information for analysis. Therefore, rewrite the previous program to print out all information about the slice again and see the result:
array := [6]int{0, 1, 2, 3, 4, 5}
The result of running this code is:
slice := array[1:3]
fmt.Println("len(slice) =", len(slice), ",cap(slice) =", cap(slice), ",slice =", slice)<br />
slice = slice[2:5]
fmt.Println("len(slice) =", len(slice), ",cap(slice) =", cap(slice), ",slice =", slice)</code></pre>
The result of running this code is:
len(slice) = 2 ,cap(slice) = 5 ,slice = [1 2]
len(slice) = 3 ,cap(slice) = 3 ,slice = [3 4 5]
Originally, after the first step of extracting a part of the array into a slice, the new slice’s capacity was already 5, not the superficially apparent length of 2! Extracting 3 elements from a total capacity of 5 in the slice is, of course, no problem! So, how exactly is the data stored for this slice generated from the array? Explaining with a diagram makes it clearer:
The yellow block in the diagram represents the original array’s allocated space in memory, with a total of 6 elements from 0 to 5. The purple represents the slice’s length, and pink represents the slice’s capacity. The two large sections represent the operations of extracting a sub-slice from an array and extracting a sub-slice from a slice, respectively. In the initial operation:
slice := array[1:3]
The slice represents the starting address slice[0]’s pointer address pointing to the memory address of array[1], and then recalculates and updates the slice’s length and capacity in the related data fields of the slice. The calculation of the length part is straightforward; when using the [i:j] syntax, the new slice’s length is already decided to be j-i. In this example, it’s 3-1, so the length is 2. The capacity calculation is from the address pointed to by slice[0] all the way to the maximum value allocated initially for the array, which is cap(array)-i. In this example, it’s 6-1, so the capacity is 5.
Understanding the logic behind this part of the operation, the following operation now looks clear:
slice = slice[2:5]
Because the internal pointer of the slice has been redirected to point to the memory address where the array’s values are located, the current slice[0] is equivalent to the previous array[1]. So, this expression can be replaced with equivalent code as follows:
// slice[0] = array[1], therefore slice[2:5] = array[3:6]
slice = array[3:6]
Since the expression is equivalent to extracting all elements within the range 3 <= {x} < 6 from the original array, the final content collection of the slice will, of course, be the elements 3, 4, 5. Writing the process of getting a sub-slice from the array as a template expression is as follows:
slice := array[i:j]
len(slice) = j - i
cap(slice) = cap(array) - i
If i and j are not explicitly assigned, like in the [:] syntax that uses all elements, how should the template expression be written? It’s simple, because the [:] shorthand syntax actually means [0:len(array)], inserting i and j gives i = 0, j = len(array). Applying it to the template expression above results in:
slice := array[:] // slice := array[0:len(array)]
len(slice) = len(array) // len(array) - 0
cap(slice) = cap(array) // cap(array) - 0
Since the size of an array must be explicitly declared at declaration time and cannot be changed, the length and capacity of an array are the same and fixed. Therefore, even if len(array) or cap(array) are used interchangeably in this segment of code, the result remains the same. Here, the length of the slice is calculated by subtracting the starting address from len(array), and the capacity of the slice is calculated by subtracting the starting address from cap(array), aiming to make the language’s statements more explicitly expressive.
Finally, summarizing the key points of the slice type:
- A pointer pointing to the first element in memory that the slice refers to
- An int representing the length of the slice, equal to the number of elements it contains
- An int representing the capacity of the slice, equal to the number of elements contained in the memory pointed to by the slice minus the index of the starting element of the slice
After this section, understanding the slice type in Go should be clear. Next, let’s return to the behavior of slices in functions and analyze why the previous examples had issues.
The Truth Revealed
Returning to the previous function code examples, let’s now understand them line by line:
// Declare a slice named other_slice
var other_slice = []int{5, 6, 7, 8, 9}
// Accepts a slice parameter
func sliceChange(a []int) {
// Since it accepts a value, the function makes a value copy, so the internal a is actually a', hence the address is different!
// The next line a = other_slice, actually means a' = other_slice!
a = other_slice
}
// Declare a slice named a
a := []int{0, 1, 2, 3, 4}
// Because a slice is a composite structure type, it's passed by value here!
sliceChange(a)
Although a slice is passed by value into the function, the memory address pointed to by the internal ptr of the slice structure is still correct. To verify this, rewrite the code as follows:
var other_slice = []int{5, 6, 7, 8, 9}
func sliceChange(a []int) {
fmt.Printf("Before resetting a, &a = %v, %p\n", a, &a)
// Directly operate on the elements inside the slice
for i := 0; i < len(a); i++ {
a[i] = other_slice[i]
}
fmt.Printf("After resetting a, &a = %v, %p\n", a, &a)
}
a := []int{0, 1, 2, 3, 4}
fmt.Printf("Before sliceChange() a, &a = %v, %p\n", a, &a)
sliceChange(a)
fmt.Printf(" After sliceChagne() a, &a = %v, %p\n", a, &a)
The execution result is:
Before sliceChange() a, &a = [0 1 2 3 4], 0x20819e020
Before resetting a, &a = [0 1 2 3 4], 0x20819e080
After resetting a, &a = [5 6 7 8 9], 0x20819e080
AFter sliceChange() a, &a = [5 6 7 8 9], 0x20819e020 // Successfully changed the values inside slice a
Indeed, it successfully changed the values inside slice a! This code also explains why slicePlusOne() was able to successfully change the values inside the slice because the operations on the slice through [i] actually reference the memory address pointed to by the internal ptr of the slice structure, so of course, there are no issues! Therefore, you can also use the method mentioned in the previous section on functions, changing the function parameter to a slice pointer, to achieve the same effect. Here is the code for this approach:
var other_slice = []int{5, 6, 7, 8, 9}
func sliceChange(a *[]int) {
*a = other_slice
}
a := []int{0, 1, 2, 3, 4}
sliceChange(&a)
Or achieve the same effect by returning the modified copy value from within the function:
var other_slice = []int{5, 6, 7, 8, 9}
func sliceChange(a []int) []int {
a = other_slice
return a
}
a := []int{0, 1, 2, 3, 4}
a = sliceChange(a)
Hands-on Practice
At this point, I believe the explanation has been quite clear. So, when you encounter such code in the future, you’ll know how to modify it:
func myAppend(a []int, e int) {
a = append(a, e) // actually, it's a' = append(a', 3)
fmt.Println(a) // a = [0 1 2 3], actually, a' = [0 1 2 3]
}
a := []int{0, 1, 2}
fmt.Println(a) // a = [0 1 2]
myAppend(a, 3)
fmt.Println(a) // a = [0 1 2]