1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
|
/*
Generic forward-only iterator that is safe and leak-free.
This package is intended to support forward-only iteration in a variety of use
cases while avoiding the normal errors and leaks that can happen with iterators
in Go. It provides mechanisms for map/select filtering, background iteration
through a goroutine, and error handling throughout.
The type of the iterator is interface{}, so it can store anything, at the cost
that you have to cast it back out when you use it. This package can be used as
is, or used as an example for creating your own forward-only iterators of more
specific types.
sum := 0
iterator.Each(func(item interface{}) {
sum = sum + item.(int)
})
Motivation
With the lack of generics and a builtin iterator pattern, iterators have been
the topic of much discussion in Go. Here are the discussions that inspired this:
http://ewencp.org/blog/golang-iterators/: Ewan Cheslack-Postava's
discussion of the major iteration patterns. Herein we have chosen the closure
pattern for iterator implementation, and given the choice of callback and
channel patterns for iteration callers.
http://blog.golang.org/pipelines: A March 2014 discussion of pipelines
on the go blog presents some of the pitfalls of channel iteration, suggesting
the "done" channel implementation to compensate.
Creating Iterators
Simple error- and cleanup-free iterators can be easily created:
// Create a simple iterator from a function
val := 1
iterator := iter.NewSimple(func() interface{} {
val = val * 2;
return val > 2000 ? val : nil // nil terminates iteration
})
Typically you will create iterators in packages ("please iterate over this
complicated thing"). You will often handle errors and have cleanup to do.
iter supports both of these. You can create a fully-functional iterator thusly:
// Create a normal iterator parsing a file, close when done
func ParseStream(reader io.ReadCloser) iter.Iterator {
return iter.Iterator{
Next: func() (iterator{}, error) {
item, err := Parse()
if item == nil && err == nil {
return nil, iter.FINISHED
}
return item, err
},
Close: func() {
reader.Close()
}
}
}
Iterating
Callback iteration looks like this and handles any bookkeeping automatically:
// Iterate over all values
err := iterator.Each(func(item interface{}) {
fmt.Println(item)
})
Sometimes you need to handle errors:
// Iterate over all values, terminating if processing has a problem
var files []*File
err := iterator.EachWithError(func(item interface{}) error {
file, err = os.Open(item.(string))
if err == nil {
files = append(files, file)
}
return err
})
Raw iteration looks like this:
defer iterator.Close() // allow the iterator to clean itself up
item, err := iterator.Next()
for err == nil {
... // do stuff with value
item, err = iterator.Next()
}
if err != iter.FINISHED {
... // handle error
}
Background goroutine iteration (using channels) deserves special mention:
// Produce the values in a goroutine, cleaning up safely afterwards.
// This allows background iteration to continue at its own pace while we
// perform blocking operations in the foreground.
var responses []http.Response
err := iterator.BackgroundEach(1000, func(item interface{}) error {
response, err := http.Get(item.(string))
if err == nil {
responses = append(list, response)
}
return err
})
Utilities
There are several useful functions provided to work with iterators:
// Square the ints
squaredIterator, err := iterator.Map(func(item interface{}) interface{} { item.int() * 2 })
// Select non-nil values
nonNilIterator, err := iterator.Select(func(item interface{}) bool) { item != nil })
// Produce a list
list, err := iterator.ToList()
*/
package iter
|
