File: metrics.md

package info (click to toggle)
puppetlabs-http-client-clojure 0.9.0-1
  • links: PTS, VCS
  • area: main
  • in suites: buster
  • size: 712 kB
  • ctags: 598
  • sloc: java: 2,381; xml: 585; makefile: 28; sh: 11
file content (442 lines) | stat: -rw-r--r-- 17,698 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
# Metrics

Both the Java client and the Clojure client have [Dropwizard
Metrics](http://metrics.dropwizard.io/3.1.0/) support - they both accept as an
option a `MetricRegistry` to which they will register http metrics for each
request, as well as metrics for any metric-ids specified in the request
options. This support is experimental - names of metrics and the exact API may
change.

For using metrics with either the Java client or the Clojure client you must
already have created a Dropwizard `MetricRegistry`.

- [Metric namespace](#metric-namespace)
- [Types of metrics](#types-of-metrics)
- [Getting back metrics](#getting-back-metrics)
- [Clojure API](#clojure-api)
  - [Creating a client with metrics](#creating-a-client-with-metrics)
  - [Setting a metric-id](#setting-a-metric-id)
  - [Filtering metrics](#filtering-metrics)
    - [Filtering by URL](#filtering-by-url)
    - [Filtering by URL and method](#filtering-by-url-and-method)
    - [Filtering by metric-id](#filtering-by-metric-id)
- [Java API](#java-api)
  - [Creating a client with metrics](#creating-a-client-with-metrics-1)
  - [Setting a metric-id](#setting-a-metric-id-1)
  - [Filtering metrics](#filtering-metrics-1)
    - [Filtering by URL](#filtering-by-url-1)
    - [Filtering by URL and method](#filtering-by-url-and-method-1)
    - [Filtering by metric-id](#filtering-by-metric-id-1)


## Metric namespace

By default, http metrics are prefixed with the namespace
`puppetlabs.http-client.experimental`. This namespace can be customized with two
client options, `server-id` and `metric-prefix`.

When `server-id` is set, the metric namespace becomes
`puppetlabs.<server-id>.http-client.experimental`.

When `metric-prefix` is set, the metric namespace becomes
`<metric-prefix>.http-client.experimental`.

If both `server-id` and `metric-prefix` are set, `metric-prefix` wins out and
a warning message is logged.

For a Clojure client, the `get-client-metric-namespace` protocol method can
be used to get back the metric namespace set for the client.

For a Java client, the `getMetricNamespace` method can be used to get back the
configured metric namespace.

## Types of metrics

There are two types of metrics: *full response* and *initial response*. Full
response metrics stop when all bytes of the response have been read by the
client. Initial response metrics stop when the first byte of the response has
been received by the client. Full response metrics are suffixed with
`full-response`. Initial response metric support has not yet been
implementedare suffixed with `full-response`. Initial response metric support
has not yet been implemented.

There are three categories of metrics: `url` metrics, `url-and-method`
metrics, and `metric-id` metrics. `url` and `url-and-method` metrics are
created automatically for each request; `metric-id` metrics are only created
for requests that have a `metric-id` request option specified.

Each http request will have a metric name created for its url (stripped of
query strings and url fragments), as well as for the url + method name. `url`
metrics have `with-url` after the prefix, followed by the url.
`url-and-method` metrics have `with-url-and-method` after the prefix, followed
by the url and then the capitalized HTTP method.

So, for example,  a `GET` request to `http://foobar.com` would create a metric
`puppetlabs.http-client.experimental.with-url.http://foobar.com.full-response`
and a metric
`puppetlabs.http-client.experimental.with-url-and-method.http://foobar.com.GET.full-response`.

It is also possible to give a request a `metric-id`. The `metric-id` is an
array of strings.  For each element in the array, a metric name will be
created, appending to the previous elements. `metric-id` metrics have
`with-metric-id` after the metric prefix, followed by the metric-id.

So, for example, for a metric-id `["foo", "bar", "baz"]`, the metrics
`puppetlabs.http-client.experimental.with-metric-id.foo.full-response`,
`puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response`,
and
`puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response`
would be created.

## Getting back metrics

Both the Clojure API and the Java API have functions to get back from a
`MetricRegistry` either the `Timer` objects or a selection of metric data, and
to filter this information based on url, url and method, or metric-id.

There are three different `Timer` objects: `UrlClientTimer` (which includes a
field for the url and `getUrl` method), `UrlAndMethodClientTimer` (which
includes a field for the url and a field for the method and accompanying
`getUrl` and `getMethod` methods), and `MetricIdClientTimer` (which includes a
field for the metric-id and accompanying `getMetricId` method). Each of these
timers also has an `isCategory` method that returns whether the timer is or is
not the provided `MetricCategory`.

Both the Clojure and Java APIs also include functions that return a list of
metrics data.  This metrics data includes the `metric-name`, `count`, `mean`
(in ms), and `aggregate` (computed as `count * mean`) for each metric. In
addition, for `url` metrics the accompanying metrics data includes the `url`,
for `url-and-method` metrics it includes the `url` and `method`, and for
`metric-id` metrics it includes the `metric-id`.

The Java API returns `ClientMetricData` objects - of which there are three
types - `UrlClientMetricData`, `UrlAndMethodClientMetricData`, and
`MetricIdClientMetricData`. The Clojure API returns maps with keys for this
information.

Both APIs have functions for returning all metrics/metric data, for returning
all metrics/metrics data from a specific category, and for filtering
metrics/metrics data within a category.

## Clojure API

### Creating a client with metrics
To use metrics with the Clojure client, pass a `MetricRegistry` to it as part
of the options map:

```clojure
(async/create-client {:metric-registry (MetricRegistry.)})
```

(the same works with the `sync` client).

In [Trapperkeeper](https://github.com/puppetlabs/trapperkeeper) applications,
creating and managing a `MetricRegistry` can be done easily with
[trapperkeeper-metrics](https://github.com/puppetlabs/trapperkeeper-metrics):

```clojure
(defservice my-trapperkeeper-service
  MyService
  [[:MetricsService get-metrics-registry]]
  (init [this context]
    (let [registry (get-metrics-registry)
          client (async/create-client {:metric-registry registry})]
      ...)))
```

Any client that is created with a `MetricRegistry` will automatically have
`url` and `url-and-method` metrics registered.

`get-client-metric-registry` protocol function can be called on a client to
get the `MetricRegistry` from it.

### Setting a metric-id

In addition to the `url` and `url-and-method` metrics, it is possible to set a
`metric-id` for a request that will create additional metrics.

For the Clojure API, a `metric-id` is a vector of keywords or strings. Either
is supported by the API, however, if special characters are needed, use
strings.  Note than even when specified as a vector of keywords in the request
option, the metric-id will be returned as a vector of strings in the metrics
data.

To set a `metric-id` for a request, include it as an option in the request
options map.

```clojure
(common/get client "https://foobar.com" {:metric-id [:foo :bar :baz]})
(common/get client "https://foobar.com" {:metric-id ["f/o/o"]})
```

### Filtering metrics

To get all `Timer` objects registered for a `MetricRegistry`, use the
`get-client-metrics` function in the `metrics` namespace.  This takes the
`MetricRegistry` as an argument and returns a map with three keys: `:url`
`:url-and-method`, and `:metric-id`. Under each of these keys is a sequence of
`Timer` objects of the corresponding type. If there are no timers of a certain
type, the sequence will be empty. The output of this function conforms to the
`common/AllTimers` schema.

Example:

```clojure
(common/get client "http://test.com" {:metric-id [:foo :bar :baz]})
...
(metrics/get-client-metrics metric-registry)
=>
{:url [#object[com.puppetlabs.http.client.metrics.UrlClientTimer
               0x66cf1f05
               "com.puppetlabs.http.client.metrics.UrlClientTimer@66cf1f05"]]
 :url-and-method [#object[com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer
                          0x6fe5444c
                          "com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer@6fe5444c"]]
 :metric-id [#object[com.puppetlabs.http.client.metrics.MetricIdClientTimer
                     0x690c10c5
                     "com.puppetlabs.http.client.metrics.MetricIdClientTimer@690c10c5"]
             #object[com.puppetlabs.http.client.metrics.MetricIdClientTimer
                     0xb7aca2e
                     "com.puppetlabs.http.client.metrics.MetricIdClientTimer@b7aca2e"]
             #object[com.puppetlabs.http.client.metrics.MetricIdClientTimer
                     0x4ef82829
                     "com.puppetlabs.http.client.metrics.MetricIdClientTimer@4ef82829"]]}
```

To get metric data for all `Timer`s registered on a `MetricRegistry`, use the
`get-client-metrics-data` function. This takes the `MetricRegistry` and returns
a map with `:url`, `:url-and-method`, and `:metric-id` as keys. Under each of
these keys is a sequence of maps, each map containing metrics data (see
[Getting back metrics](#Getting-back-metrics) above, conforming to the
`common/AllMetricsData` schema.

Example:

```clojure
(common/get client "http://test.com" {:metric-id [:foo :bar :baz]})
...
(metrics/get-client-metrics-data metric-registry)
=>
{:url ({:count 1
        :mean 553
        :aggregate 553
        :metric-name "puppetlabs.http-client.experimental.with-url.http://test.com.full-response"
        :url "http://test.com"})
 :url-and-method ({:count 1
                   :mean 554
                   :aggregate 554
                   :metric-name "puppetlabs.http-client.experimental.with-url-and-method.http://test.com.GET.full-response"
                   :url "http://test.com"
                   :method "GET"})
 :metric-id ({:count 1
              :mean 554
              :aggregate 554
              :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response"
              :metric-id ["foo" "bar" "baz"]}
             {:count 1
              :mean 554
              :aggregate 554
              :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response"
              :metric-id ["foo" "bar"]}
             {:count 1
              :mean 554
              :aggregate 554
              :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.full-response"
              :metric-id ["foo"]})}
```

#### Filtering by URL

To get URL metrics and metrics data, use the `get-client-metrics-by-url`
function and `get-client-metrics-data-by-url` in the `metrics` namespace.

Both of these take as arguments the `MetricRegistry` and a string url. If no
url is provided, return all url metrics/metrics data in a sequence. If no
metrics are registered for that url, return an empty sequence.

Example:

```clojure
(common/get client "http://test.com" {:metric-id [:foo :bar :baz]})
...
(metrics/get-client-metrics-data-by-url metric-registry "http://test.com")
=>
({:count 1
  :mean 553
  :aggregate 553
  :metric-name "puppetlabs.http-client.experimental.with-url.http://test.com.full-response"
  :url "http://test.com"})

(metrics/get-client-metrics-data-by-url metric-registry "http://not-a-matching-url.com")
=> ()
```

#### Filtering by URL and method

To get URL and method metrics and metrics data, use the
`get-client-metrics-by-url-and-method` and
`get-client-metrics-data-by-url-and-method` functions in the `metrics`
namespace.

Both of these take as arguments the `MetricRegistry`, a string url, and a
keyword HTTP method. If no url and method is provided, return all url
metrics/metrics data in a sequence. If no metrics are registered for that url
and method, return an empty sequence.

Example:

```clojure
(common/get client "http://test.com" {:metric-id [:foo :bar :baz]})
...
(metrics/get-client-metrics-data-by-url-and-method metric-registry "http://test.com" :get)
=>
({:count 1
  :mean 554
  :aggregate 554
  :metric-name "puppetlabs.http-client.experimental.with-url-and-method.http://test.com.GET.full-response"
  :url "http://test.com"
  :method "GET"})

  :method "GET"})

(metrics/get-client-metrics-data-by-url-and-method metric-registry "http://test.com" :post)
=> ()
```

#### Filtering by metric-id

To get metric-id metrics and metrics data, use the
`get-client-metrics-by-metric-id` and `get-client-metrics-data-by-metric-id`
functions in the `metrics` namespace.

Both of these take as arguments the `MetricRegistry` and a metric-id - as a
vector of keywords or strings (if special characters are needed, use strings).
If no metric-id is provided, will return all metric-id metrics.  If no metrics
are registered for that metric-id, returns an empty list.

Example:

```clojure
(common/get client "http://test.com" {:metric-id [:foo :bar :baz]})
...
(metrics/get-client-metrics-data-by-metric-id metric-registry [:foo])
=>
({:count 1
  :mean 554
  :aggregate 554
  :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.full-response"
  :metric-id ["foo"]})

(metrics/get-client-metrics-data-by-metric-id metric-registry [:foo :bar])
=>
({:count 1
  :mean 554
  :aggregate 554
  :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response"
  :metric-id ["foo" "bar"]})

(metrics/get-client-metrics-data-by-metric-id metric-registry [:foo :nope])
=> ()
```

## Java API

### Creating a client with metrics
To use metrics with the Java client, call `setMetricRegistry()` on the
`ClientOptions` before supplying them to create the client.

```java
MetricRegistry registry = new MetricRegistry();
ClientOptions options = new ClientOptions();
AsyncHttpClient client = Async.createClient(options);
```

(the same works with the `SyncHttpClient`).

Any client that is created with a `MetricRegistry` will automatically have
`url` and `url-and-method` metrics registered.

`getMetricRegistry()` can be called on a client to get the `MetricRegistry`
from it.

### Setting a metric-id

In addition to the `url` and `url-and-method` metrics, it is possible to set a
`metric-id` for a request that will create additional metrics.

A `metric-id` is an array of strings.

To set a `metric-id` for a request, use the `.setMetricId` method on the
`RequestOptions` class.

```java
RequestOptions options = new RequestOptions("https://foobar.com");
options.setMetricId(["foo", "bar", "baz"]);
client.get(options);
```

`getMetricId()` can be called on `RequestOptions` to get back the `metric-id`.

### Filtering metrics

To get all `Timer` objects registered for a `MetricRegistry`, use the
`getClientMetrics()` method in the `Metrics` class. This takes the
`MetricRegistry` as an argument and returns a `ClientTimerContainer` object.

The `ClientTimerContainer` object has three fields - `urlTimers`,
`urlAndMethodTimers`, and `metricIdTimers`. The list of `URLClientTimers` can
be retrieved from the `ClientTimerContainer` with the `getUrlTimers()` method.
A list of `URlAndMethodClientTimers` can be retrieved with the
`getUrlAndMethod()` method. A list of `MetricIdClientTimer`s can be retrieved
with the `getMetricIdTimers()` method.

To get all `MetricData` objects, representing data for each `Metric`
registered for a `MetricRegistry`, use the `getClientMetricsData()` methods in
the `Metric` class. This takes the `MetricRegistry` as an argument and returns
a `ClientMetricDataContainer` object. This object has a field for each type of
metric data - `urlData`, `urlAndMethodData`, and `metricIdData`. Each field
has an associated getter returning a list of the appropriate data object -
`UrlClientMetricData`, `UrlAndMethodClientMetricData`, and
`MetricIdClientMetricData`.

#### Filtering by URL

To get URL metrics and metrics data, use the `getClientMetricsByUrl()` and
`getClientMetricsDataByUrl()` methods in the `Metric` class.

Both of these take as arguments the `MetricRegistry` and a string url. If no
url is provided, return all url metrics. If no metrics are registered for that
url, return an empty list.

`getClientMetricsByUrl()` returns a list of `UrlClientTimer` objects.
`getClientMetricsDataByUrl` returns a list of `UrlClientMetricData` objects.

#### Filtering by URL and method

To get URL and method metrics and metrics data, use the
`getClientMetricsByUrlAndMethod()` and `getClientMetricsDataByUrlAndMethod()`
methods in the `Metric` class.

Both of these take as arguments the `MetricRegistry`, a string url, and a
string HTTP method. If no url or method is provided, will return all
url-and-method metrics. If no metrics are registered for that url and method,
returns an empty list.

`getClientMetricsByUrlAndMethod()` returns a list of `UrlAndMethodClientTimer`
objects. `getClientMetricsDataByUrlAndMethod` returns a list of
`UrlAndMethodClientMetricData` objects.

#### Filtering by metric-id

To get metric-id metrics and metrics data, use the
`getClientMetricsByMetricId()` and `getClientMetricsDataByMetricId()` methods
in the `Metric` class.

Both of these take as arguments the `MetricRegistry` and a metric-id - as an
array of strings. If no metric-id is provided, will return all metric-id
metrics. If no metrics are registered for that metric-id, returns an empty
list.

`getClientMetricsByMetricId()` returns a list of `MetricIdClientTimer`
objects. `getClientMetricsDataByMetricId()` returns a list of
`MetricIdClientMetricData` objects.