File: README.md

package info (click to toggle)
specter-clojure 1.0.2-2
  • links: PTS, VCS
  • area: main
  • in suites: bullseye, buster, sid
  • size: 360 kB
  • ctags: 12
  • sloc: xml: 113; makefile: 21; java: 19; sh: 3
file content (310 lines) | stat: -rw-r--r-- 12,688 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
# Specter [![Build Status](https://secure.travis-ci.org/nathanmarz/specter.png?branch=master)](http://travis-ci.org/nathanmarz/specter)

Clojure has fantastic facilities for doing immutable programming, with a rich library of persistent data structures and efficient mechanisms for manipulating and traversing them. However, Clojure's story is incomplete. Once you nest data structures – which is extremely common – Clojure becomes cumbersome and complex. Clojure even lacks a facility for a basic task like transforming every value in a generic sequence without changing the type or order of that sequence.

Specter, available for both Clojure and ClojureScript, provides a high performance abstraction called navigators which complete the story around immutable programming and make it easy to transform and query nested data structures. It allows you to concisely specify what you want to change within a data structure, and get a new data structure back with only your changes applied – everything else is reconstructed and the types of data structures throughout don't unexpectedly change.

Consider these examples:

**Example 1: Increment every even number nested within map of vector of maps**

```clojure
(def data {:a [{:aa 1 :bb 2}
               {:cc 3}]
           :b [{:dd 4}]})

;; Manual Clojure
(defn map-vals [m afn]
  (->> m (map (fn [[k v]] [k (afn v)])) (into {})))

(map-vals data
  (fn [v]
    (mapv
      (fn [m]
        (map-vals
          m
          (fn [v] (if (even? v) (inc v) v))))
      v)))

;; Specter
(transform [MAP-VALS ALL MAP-VALS even?] inc data)
```

**Example 2: Append a sequence of elements to a nested vector**

```clojure
(def data {:a [1 2 3]})

;; Manual Clojure
(update data :a (fn [v] (into (if v v []) [4 5])))

;; Specter
(setval [:a END] [4 5] data)
```

**Example 3: Increment the last odd number in a sequence**

```clojure
(def data [1 2 3 4 5 6 7 8])

;; Manual Clojure
(let [idx (reduce-kv (fn [res i v] (if (odd? v) i res)) nil data)]
  (if idx (update data idx inc) data))

;; Specter
(transform [(filterer odd?) LAST] inc data)
```

**Example 4: Map a function over a sequence without changing the type or order of the sequence**

```clojure
;; Manual Clojure
(map inc data) ;; doesn't work, becomes a lazy sequence
(into (empty data) (map inc data)) ;; doesn't work, reverses the order of lists

;; Specter
(transform ALL inc data) ;; works for all Clojure datatypes with near-optimal efficiency
```


Specter has performance rivaling hand-optimized code  (see [this benchmark](https://gist.github.com/nathanmarz/b7c612b417647db80b9eaab618ff8d83)). Clojure's only comparable built-in operations are `get-in` and `update-in`, and the Specter equivalents are 30% and 85% faster respectively (while being just as concise). Under the hood, Specter uses [advanced dynamic techniques](https://github.com/nathanmarz/specter/wiki/Specter's-inline-caching-implementation) to strip away the overhead of composition. Additionally, the built-in navigators use the most efficient means possible of accessing data structures. For example, `ALL` uses `mapv` on vectors, the `IMapIterable` interface on small maps, and `reduce-kv` in conjunction with transients on larger maps.

The most important aspect of Specter is its composability. Specter navigators can be composed with any other navigators, so the supported use cases grow combinatorially. And because Specter is completely extensible, it can be used to navigate any data structure or object you have.


# Latest Version

The latest release version of Specter is hosted on [Clojars](https://clojars.org):

[![Current Version](https://clojars.org/com.rpl/specter/latest-version.svg)](https://clojars.org/com.rpl/specter)

# Learn Specter

- Introductory blog post: [Clojure's missing piece](http://nathanmarz.com/blog/clojures-missing-piece.html)
- Presentation about Specter: [Specter: Powerful and Simple Data Structure Manipulation](https://www.youtube.com/watch?v=VTCy_DkAJGk)
  - Note that this presentation was given before Specter's inline compilation/caching system was developed. You no longer need to do anything special to get near-optimal performance.
- List of navigators with examples: [This wiki page](https://github.com/nathanmarz/specter/wiki/List-of-Navigators) provides a more comprehensive overview than the API docs about the behavior of specific navigators and includes many examples.
- Core operations and defining new navigators: [This wiki page](https://github.com/nathanmarz/specter/wiki/List-of-Macros) provides a more comprehensive overview than the API docs of the core select/transform/etc. operations and the operations for defining new navigators.
- [API docs](http://nathanmarz.github.io/specter/)
- Performance guide: [This post](https://github.com/nathanmarz/specter/wiki/Specter's-inline-caching-implementation) provides an overview of how Specter achieves its performance.

Specter's API is contained in these files:

- [specter.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter.cljc): This contains the built-in navigators and the definition of the core operations.
- [transients.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/transients.cljc): This contains navigators for transient collections.
- [zipper.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/zipper.cljc): This integrates zipper-based navigation into Specter.

# Questions?

You can ask questions about Specter by [opening an issue](https://github.com/nathanmarz/specter/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Aquestion+) on Github.

You can also find help in the #specter channel on [Clojurians](http://clojurians.net/).

# Examples

Increment all the values in maps of maps:
```clojure
user> (use 'com.rpl.specter)
user> (transform [MAP-VALS MAP-VALS]
              inc
              {:a {:aa 1} :b {:ba -1 :bb 2}})
{:a {:aa 2}, :b {:ba 0, :bb 3}}
```

Increment all the even values for :a keys in a sequence of maps:

```clojure
user> (transform [ALL :a even?]
              inc
              [{:a 1} {:a 2} {:a 4} {:a 3}])
[{:a 1} {:a 3} {:a 5} {:a 3}]
```

Retrieve every number divisible by 3 out of a sequence of sequences:
```clojure
user> (select [ALL ALL #(= 0 (mod % 3))]
              [[1 2 3 4] [] [5 3 2 18] [2 4 6] [12]])
[3 3 18 6 12]
```

Increment the last odd number in a sequence:

```clojure
user> (transform [(filterer odd?) LAST]
              inc
              [2 1 3 6 9 4 8])
[2 1 3 6 10 4 8]
```

Remove nils from a nested sequence:

```clojure
user> (setval [:a ALL nil?] NONE {:a [1 2 nil 3 nil]})
{:a [1 2 3]}
```

Increment all the odd numbers between indices 1 (inclusive) and 4 (exclusive):

```clojure
user> (transform [(srange 1 4) ALL odd?] inc [0 1 2 3 4 5 6 7])
[0 2 2 4 4 5 6 7]
```

Replace the subsequence from indices 2 to 4 with [:a :b :c :d :e]:

```clojure
user> (setval (srange 2 4) [:a :b :c :d :e] [0 1 2 3 4 5 6 7 8 9])
[0 1 :a :b :c :d :e 4 5 6 7 8 9]
```

Concatenate the sequence [:a :b] to every nested sequence of a sequence:

```clojure
user> (setval [ALL END] [:a :b] [[1] '(1 2) [:c]])
[[1 :a :b] (1 2 :a :b) [:c :a :b]]
```

Get all the numbers out of a data structure, no matter how they're nested:

```clojure
user> (select (walker number?)
              {2 [1 2 [6 7]] :a 4 :c {:a 1 :d [2 nil]}})
[2 1 2 1 2 6 7 4]
```

Navigate via non-keyword keys:

```clojure
user> (select (keypath "a" "b")
              {"a" {"b" 10}})
[10]
```

Reverse the positions of all even numbers between indices 4 and 11:

```clojure
user> (transform [(srange 4 11) (filterer even?)]
              reverse
              [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15])
[0 1 2 3 10 5 8 7 6 9 4 11 12 13 14 15]
```

Append [:c :d] to every subsequence that has at least two even numbers:
```clojure
user> (setval [ALL
               (selected? (filterer even?) (view count) (pred>= 2))
               END]
              [:c :d]
              [[1 2 3 4 5 6] [7 0 -1] [8 8] []])
[[1 2 3 4 5 6 :c :d] [7 0 -1] [8 8 :c :d] []]
```

When doing more involved transformations, you often find you lose context when navigating deep within a data structure and need information "up" the data structure to perform the transformation. Specter solves this problem by allowing you to collect values during navigation to use in the transform function. Here's an example which transforms a sequence of maps by adding the value of the :b key to the value of the :a key, but only if the :a key is even:

```clojure
user> (transform [ALL (collect-one :b) :a even?]
              +
              [{:a 1 :b 3} {:a 2 :b -10} {:a 4 :b 10} {:a 3}])
[{:b 3, :a 1} {:b -10, :a -8} {:b 10, :a 14} {:a 3}]
```

The transform function receives as arguments all the collected values followed by the navigated to value. So in this case `+` receives the value of the :b key followed by the value of the :a key, and the transform is performed to :a's value.

The four built-in ways for collecting values are `VAL`, `collect`, `collect-one`, and `putval`. `VAL` just adds whatever element it's currently on to the value list, while `collect` and `collect-one` take in a selector to navigate to the desired value. `collect` works just like `select` by finding a sequence of values, while `collect-one` expects to only navigate to a single value. Finally, `putval` adds an external value into the collected values list.


Increment the value for :a key by 10:
```clojure
user> (transform [:a (putval 10)]
              +
              {:a 1 :b 3})
{:b 3 :a 11}
```


For every map in a sequence, increment every number in :c's value if :a is even or increment :d if :a is odd:

```clojure
user> (transform [ALL (if-path [:a even?] [:c ALL] :d)]
              inc
              [{:a 2 :c [1 2] :d 4} {:a 4 :c [0 10 -1]} {:a -1 :c [1 1 1] :d 1}])
[{:c [2 3], :d 4, :a 2} {:c [1 11 0], :a 4} {:c [1 1 1], :d 2, :a -1}]
```

"Protocol paths" can be used to navigate on polymorphic data. For example, if you have two ways of storing "account" information:

```clojure
(defrecord Account [funds])
(defrecord User [account])
(defrecord Family [accounts-list])
```

You can make an "AccountPath" that dynamically chooses its path based on the type of element it is currently navigated to:


```clojure
(defprotocolpath AccountPath [])
(extend-protocolpath AccountPath
  User :account
  Family [:accounts-list ALL])
```

Then, here is how to select all the funds out of a list of `User` and `Family`:

```clojure
user> (select [ALL AccountPath :funds]
        [(->User (->Account 50))
         (->User (->Account 51))
         (->Family [(->Account 1)
                    (->Account 2)])
         ])
[50 51 1 2]
```

The next examples demonstrate recursive navigation. Here's one way to double all the even numbers in a tree:

```clojure
(defprotocolpath TreeWalker [])

(extend-protocolpath TreeWalker
  Object nil
  clojure.lang.PersistentVector [ALL TreeWalker])

(transform [TreeWalker number? even?] #(* 2 %) [:a 1 [2 [[[3]]] :e] [4 5 [6 7]]])
;; => [:a 1 [4 [[[3]]] :e] [8 5 [12 7]]]
```

Here's how to reverse the positions of all even numbers in a tree (with order based on a depth first search). This example uses conditional navigation instead of protocol paths to do the walk:

```clojure
(def TreeValues
  (recursive-path [] p
    (if-path vector?
      [ALL p]
      STAY
      )))


(transform (subselect TreeValues even?)
  reverse
  [1 2 [3 [[4]] 5] [6 [7 8] 9 [[10]]]]
  )
;; => [1 10 [3 [[8]] 5] [6 [7 4] 9 [[2]]]]
```

# ClojureScript

Specter supports ClojureScript! However, some of the differences between Clojure and ClojureScript affect how you use Specter in ClojureScript, in particular with the namespace declarations. In Clojure, you might `(use 'com.rpl.specter)` or say `(:require [com.rpl.specter :refer :all])` in your namespace declaration. But in ClojureScript, these options [aren't allowed](https://groups.google.com/d/msg/clojurescript/SzYK08Oduxo/MxLUjg50gQwJ). Instead, consider using one of these options:

```clojure
(:require [com.rpl.specter :as s])
(:require [com.rpl.specter :as s :refer-macros [select transform]]) ;; add in the Specter macros that you need
```

# Future work
- Integrate Specter with other kinds of data structures, such as graphs. Desired navigations include: reduction in topological order, navigate to outgoing/incoming nodes, to a subgraph (with metadata indicating how to attach external edges on transformation), to node attributes, to node values, to specific nodes.
- Make it possible to parallelize selects/transforms

# License

Copyright 2015-2017 Red Planet Labs, Inc. Specter is licensed under Apache License v2.0.