File: dockerd.8.md

package info (click to toggle)
docker.io 20.10.24%2Bdfsg1-1%2Bdeb12u1
  • links: PTS, VCS
  • area: main
  • in suites: bookworm, bookworm-proposed-updates
  • size: 60,824 kB
  • sloc: sh: 5,621; makefile: 593; ansic: 179; python: 162; asm: 7
file content (849 lines) | stat: -rw-r--r-- 30,393 bytes parent folder | download | duplicates (4)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
% "DOCKERD" "8" "SEPTEMBER 2015" "Docker Community" "Docker User Manuals"

# NAME
dockerd - Enable daemon mode

# SYNOPSIS
**dockerd**
[**--add-runtime**[=*[]*]]
[**--allow-nondistributable-artifacts**[=*[]*]]
[**--api-cors-header**=[=*API-CORS-HEADER*]]
[**--authorization-plugin**[=*[]*]]
[**-b**|**--bridge**[=*BRIDGE*]]
[**--bip**[=*BIP*]]
[**--cgroup-parent**[=*[]*]]
[**--cluster-store**[=*[]*]]
[**--cluster-advertise**[=*[]*]]
[**--cluster-store-opt**[=*map[]*]]
[**--config-file**[=*/etc/docker/daemon.json*]]
[**--containerd**[=*SOCKET-PATH*]]
[**--data-root**[=*/var/lib/docker*]]
[**-D**|**--debug**]
[**--default-cgroupns-mode**[=*host*]]
[**--default-gateway**[=*DEFAULT-GATEWAY*]]
[**--default-gateway-v6**[=*DEFAULT-GATEWAY-V6*]]
[**--default-address-pool**[=*DEFAULT-ADDRESS-POOL*]]
[**--default-runtime**[=*runc*]]
[**--default-ipc-mode**=*MODE*]
[**--default-shm-size**[=*64MiB*]]
[**--default-ulimit**[=*[]*]]
[**--dns**[=*[]*]]
[**--dns-opt**[=*[]*]]
[**--dns-search**[=*[]*]]
[**--exec-opt**[=*[]*]]
[**--exec-root**[=*/var/run/docker*]]
[**--experimental**[=*false*]]
[**--fixed-cidr**[=*FIXED-CIDR*]]
[**--fixed-cidr-v6**[=*FIXED-CIDR-V6*]]
[**-G**|**--group**[=*docker*]]
[**-H**|**--host**[=*[]*]]
[**--help**]
[**--icc**[=*true*]]
[**--init**[=*false*]]
[**--init-path**[=*""*]]
[**--insecure-registry**[=*[]*]]
[**--ip**[=*0.0.0.0*]]
[**--ip-forward**[=*true*]]
[**--ip-masq**[=*true*]]
[**--iptables**[=*true*]]
[**--ipv6**]
[**--isolation**[=*default*]]
[**-l**|**--log-level**[=*info*]]
[**--label**[=*[]*]]
[**--live-restore**[=*false*]]
[**--log-driver**[=*json-file*]]
[**--log-opt**[=*map[]*]]
[**--mtu**[=*0*]]
[**--max-concurrent-downloads**[=*3*]]
[**--max-concurrent-uploads**[=*5*]]
[**--max-download-attempts**[=*5*]]
[**--node-generic-resources**[=*[]*]]
[**-p**|**--pidfile**[=*/var/run/docker.pid*]]
[**--raw-logs**]
[**--registry-mirror**[=*[]*]]
[**-s**|**--storage-driver**[=*STORAGE-DRIVER*]]
[**--seccomp-profile**[=*SECCOMP-PROFILE-PATH*]]
[**--selinux-enabled**]
[**--shutdown-timeout**[=*15*]]
[**--storage-opt**[=*[]*]]
[**--swarm-default-advertise-addr**[=*IP|INTERFACE*]]
[**--tls**]
[**--tlscacert**[=*~/.docker/ca.pem*]]
[**--tlscert**[=*~/.docker/cert.pem*]]
[**--tlskey**[=*~/.docker/key.pem*]]
[**--tlsverify**]
[**--userland-proxy**[=*true*]]
[**--userland-proxy-path**[=*""*]]
[**--userns-remap**[=*default*]]

# DESCRIPTION
**dockerd** is used for starting the Docker daemon (i.e., to command the daemon
to manage images, containers etc).  So **dockerd** is a server, as a daemon.

To run the Docker daemon you can specify **dockerd**.
You can check the daemon options using **dockerd --help**.
Daemon options should be specified after the **dockerd** keyword in the
following format.

**dockerd [OPTIONS]**

# OPTIONS

**--add-runtime**=[]
  Runtimes can be registered with the daemon either via the
configuration file or using the `--add-runtime` command line argument.

  The following is an example adding 2 runtimes via the configuration:

```json
{
	"default-runtime": "runc",
	"runtimes": {
		"runc": {
			"path": "runc"
		},
		"custom": {
			"path": "/usr/local/bin/my-runc-replacement",
			"runtimeArgs": [
				"--debug"
			]
		}
	}
}
```

  This is the same example via the command line:

```bash
$ sudo dockerd --add-runtime runc=runc --add-runtime custom=/usr/local/bin/my-runc-replacement
```

  **Note**: defining runtime arguments via the command line is not supported.

**--allow-nondistributable-artifacts**=[]
  Push nondistributable artifacts to the specified registries.

  List can contain elements with CIDR notation to specify a whole subnet.

  This option is useful when pushing images containing nondistributable
  artifacts to a registry on an air-gapped network so hosts on that network can
  pull the images without connecting to another server.

  **Warning**: Nondistributable artifacts typically have restrictions on how
  and where they can be distributed and shared. Only use this feature to push
  artifacts to private registries and ensure that you are in compliance with
  any terms that cover redistributing nondistributable artifacts.

**--api-cors-header**=""
  Set CORS headers in the Engine API. Default is cors disabled. Give urls like
  "http://foo, http://bar, ...". Give "*" to allow all.

**--authorization-plugin**=""
  Set authorization plugins to load

**-b**, **--bridge**=""
  Attach containers to a pre\-existing network bridge; use 'none' to disable
  container networking

**--bip**=""
  Use the provided CIDR notation address for the dynamically created bridge
  (docker0); Mutually exclusive of \-b

**--cgroup-parent**=""
  Set parent cgroup for all containers. Default is "/docker" for fs cgroup
  driver and "system.slice" for systemd cgroup driver.

**--cluster-store**=""
  URL of the distributed storage backend

**--cluster-advertise**=""
  Specifies the 'host:port' or `interface:port` combination that this
  particular daemon instance should use when advertising itself to the cluster.
  The daemon is reached through this value.

**--cluster-store-opt**=""
  Specifies options for the Key/Value store.

**--config-file**="/etc/docker/daemon.json"
  Specifies the JSON file path to load the configuration from.

**--containerd**=""
  Path to containerd socket.

**--data-root**=""
  Path to the directory used to store persisted Docker data such as
  configuration for resources, swarm cluster state, and filesystem data for
  images, containers, and local volumes. Default is `/var/lib/docker`.

**-D**, **--debug**=*true*|*false*
  Enable debug mode. Default is false.

**--default-cgroupns-mode**="**host**|**private**"
  Set the default cgroup namespace mode for newly created containers. The argument
  can either be **host** or **private**. If unset, this defaults to `host` on cgroup v1,
`private` on cgroup v2.

**--default-gateway**=""
  IPv4 address of the container default gateway; this address must be part of
  the bridge subnet (which is defined by \-b or \--bip)

**--default-gateway-v6**=""
  IPv6 address of the container default gateway

**--default-address-pool**=""
  Default address pool from which IPAM driver selects a subnet for the networks.
  Example: base=172.30.0.0/16,size=24 will set the default
  address pools for the selected scope networks to {172.30.[0-255].0/24}

**--default-runtime**="runc"
  Set default runtime if there're more than one specified by `--add-runtime`.

**--default-ipc-mode**="**private**|**shareable**"
  Set the default IPC mode for newly created containers. The argument
  can either be **private** or **shareable**.

**--default-shm-size**=*64MiB*
  Set the daemon-wide default shm size for containers. Default is `64MiB`.

**--default-ulimit**=[]
  Default ulimits for containers.

**--dns**=""
  Force Docker to use specific DNS servers

**--dns-opt**=""
  DNS options to use.

**--dns-search**=[]
  DNS search domains to use.

**--exec-opt**=[]
  Set runtime execution options. See RUNTIME EXECUTION OPTIONS.

**--exec-root**=""
  Path to use as the root of the Docker execution state files. Default is
  `/var/run/docker`.

**--experimental**=""
  Enable the daemon experimental features.

**--fixed-cidr**=""
  IPv4 subnet for fixed IPs (e.g., 10.20.0.0/16); this subnet must be nested in
  the bridge subnet (which is defined by \-b or \-\-bip).

**--fixed-cidr-v6**=""
  IPv6 subnet for global IPv6 addresses (e.g., 2a00:1450::/64)

**-G**, **--group**=""
  Group to assign the unix socket specified by -H when running in daemon mode.
  use '' (the empty string) to disable setting of a group. Default is `docker`.

**-H**, **--host**=[*unix:///var/run/docker.sock*]: tcp://[host:port] to bind or
unix://[/path/to/socket] to use.
  The socket(s) to bind to in daemon mode specified using one or more
  tcp://host:port, unix:///path/to/socket, fd://* or fd://socketfd.

**--help**
  Print usage statement

**--icc**=*true*|*false*
  Allow unrestricted inter\-container and Docker daemon host communication. If
  disabled, containers can still be linked together using the **--link** option
  (see **docker-run(1)**). Default is true.

**--init**
  Run an init process inside containers for signal forwarding and process
  reaping.

**--init-path**
  Path to the docker-init binary.

**--insecure-registry**=[]
  Enable insecure registry communication, i.e., enable un-encrypted and/or
  untrusted communication.

  List of insecure registries can contain an element with CIDR notation to
  specify a whole subnet. Insecure registries accept HTTP and/or accept HTTPS
  with certificates from unknown CAs.

  Enabling `--insecure-registry` is useful when running a local registry.
  However, because its use creates security vulnerabilities it should ONLY be
  enabled for testing purposes.  For increased security, users should add their
  CA to their system's list of trusted CAs instead of using
  `--insecure-registry`.

**--ip**=""
  Default IP address to use when binding container ports. Default is `0.0.0.0`.

**--ip-forward**=*true*|*false*
  Enables IP forwarding on the Docker host. The default is `true`. This flag
  interacts with the IP forwarding setting on your host system's kernel. If
  your system has IP forwarding disabled, this setting enables it. If your
  system has IP forwarding enabled, setting this flag to `--ip-forward=false`
  has no effect.

  This setting will also enable IPv6 forwarding if you have both
  `--ip-forward=true` and `--fixed-cidr-v6` set. Note that this may reject
  Router Advertisements and interfere with the host's existing IPv6
  configuration. For more information, please consult the documentation about
  "Advanced Networking - IPv6".

**--ip-masq**=*true*|*false*
  Enable IP masquerading for bridge's IP range. Default is true.

**--iptables**=*true*|*false*
  Enable Docker's addition of iptables rules. Default is true.

**--ipv6**=*true*|*false*
  Enable IPv6 support. Default is false. Docker will create an IPv6-enabled
  bridge with address fe80::1 which will allow you to create IPv6-enabled
  containers. Use together with `--fixed-cidr-v6` to provide globally routable
  IPv6 addresses. IPv6 forwarding will be enabled if not used with
  `--ip-forward=false`. This may collide with your host's current IPv6
  settings. For more information please consult the documentation about
  "Advanced Networking - IPv6".

**--isolation**="*default*"
   Isolation specifies the type of isolation technology used by containers.
   Note that the default on Windows server is `process`, and the default on
   Windows client is `hyperv`. Linux only supports `default`.

**-l**, **--log-level**="*debug*|*info*|*warn*|*error*|*fatal*"
  Set the logging level. Default is `info`.

**--label**="[]"
  Set key=value labels to the daemon (displayed in `docker info`)

**--live-restore**=*false*
  Enable live restore of running containers when the daemon starts so that they
  are not restarted. This option is applicable only for docker daemon running
  on Linux host.

**--log-driver**="*json-file*|*syslog*|*journald*|*gelf*|*fluentd*|*awslogs*|*splunk*|*etwlogs*|*gcplogs*|*none*"
  Default driver for container logs. Default is `json-file`.
  **Warning**: `docker logs` command works only for `json-file` logging driver.

**--log-opt**=[]
  Logging driver specific options.

**--mtu**=*0*
  Set the containers network mtu. Default is `0`.

**--max-concurrent-downloads**=*3*
  Set the max concurrent downloads. Default is `3`.

**--max-concurrent-uploads**=*5*
  Set the max concurrent uploads. Default is `5`.

**--max-download-attempts**=*5*
  Set the max download attempts for each pull. Default is `5`.

**--node-generic-resources**=*[]*
  Advertise user-defined resource. Default is `[]`.
  Use this if your swarm cluster has some nodes with custom
  resources (e.g: NVIDIA GPU, SSD, ...) and you need your services to land on
  nodes advertising these resources.
  Usage example: `--node-generic-resources "NVIDIA-GPU=UUID1"
  --node-generic-resources "NVIDIA-GPU=UUID2"`


**-p**, **--pidfile**=""
  Path to use for daemon PID file. Default is `/var/run/docker.pid`

**--raw-logs**
  Output daemon logs in full timestamp format without ANSI coloring. If this
  flag is not set, the daemon outputs condensed, colorized logs if a terminal
  is detected, or full ("raw") output otherwise.

**--registry-mirror**=*<scheme>://<host>*
  Prepend a registry mirror to be used for image pulls. May be specified
  multiple times.

**-s**, **--storage-driver**=""
  Force the Docker runtime to use a specific storage driver.

**--seccomp-profile**=""
  Path to seccomp profile.

**--selinux-enabled**=*true*|*false*
  Enable selinux support. Default is false.

**--shutdown-timeout**=*15*
  Set the shutdown timeout value in seconds. Default is `15`.

**--storage-opt**=[]
  Set storage driver options. See STORAGE DRIVER OPTIONS.

**--swarm-default-advertise-addr**=*IP|INTERFACE*
  Set default address or interface for swarm to advertise as its
  externally-reachable address to other cluster members. This can be a
  hostname, an IP address, or an interface such as `eth0`. A port cannot be
  specified with this option.

**--tls**=*true*|*false*
  Use TLS; implied by --tlsverify. Default is false.

**--tlscacert**=*~/.docker/ca.pem*
  Trust certs signed only by this CA.

**--tlscert**=*~/.docker/cert.pem*
  Path to TLS certificate file.

**--tlskey**=*~/.docker/key.pem*
  Path to TLS key file.

**--tlsverify**=*true*|*false*
  Use TLS and verify the remote (daemon: verify client, client: verify daemon).
  Default is false.

**--userland-proxy**=*true*|*false*
  Rely on a userland proxy implementation for inter-container and
  outside-to-container loopback communications. Default is true.

**--userland-proxy-path**=""
  Path to the userland proxy binary.

**--userns-remap**=*default*|*uid:gid*|*user:group*|*user*|*uid*
  Enable user namespaces for containers on the daemon. Specifying "default"
  will cause a new user and group to be created to handle UID and GID range
  remapping for the user namespace mappings used for contained processes.
  Specifying a user (or uid) and optionally a group (or gid) will cause the
  daemon to lookup the user and group's subordinate ID ranges for use as the
  user namespace mappings for contained processes.

# STORAGE DRIVER OPTIONS

Docker uses storage backends (known as "graphdrivers" in the Docker
internals) to create writable containers from images.  Many of these
backends use operating system level technologies and can be
configured.

Specify options to the storage backend with **--storage-opt** flags. The
backends that currently take options are *devicemapper*, *zfs* and *btrfs*.
Options for *devicemapper* are prefixed with *dm*, options for *zfs*
start with *zfs* and options for *btrfs* start with *btrfs*.

Specifically for devicemapper, the default is a "loopback" model which
requires no pre-configuration, but is extremely inefficient.  Do not
use it in production.

To make the best use of Docker with the devicemapper backend, you must
have a recent version of LVM.  Use `lvm` to create a thin pool; for
more information see `man lvmthin`.  Then, use `--storage-opt
dm.thinpooldev` to tell the Docker engine to use that pool for
allocating images and container snapshots.

## Devicemapper options

#### dm.thinpooldev

Specifies a custom block storage device to use for the thin pool.

If using a block device for device mapper storage, it is best to use `lvm`
to create and manage the thin-pool volume. This volume is then handed to Docker
to exclusively create snapshot volumes needed for images and containers.

Managing the thin-pool outside of Engine makes for the most feature-rich
method of having Docker utilize device mapper thin provisioning as the
backing storage for Docker containers. The highlights of the lvm-based
thin-pool management feature include: automatic or interactive thin-pool
resize support, dynamically changing thin-pool features, automatic thinp
metadata checking when lvm activates the thin-pool, etc.

As a fallback if no thin pool is provided, loopback files are
created. Loopback is very slow, but can be used without any
pre-configuration of storage. It is strongly recommended that you do
not use loopback in production. Ensure your Engine daemon has a
`--storage-opt dm.thinpooldev` argument provided.

Example use:

   $ dockerd \
         --storage-opt dm.thinpooldev=/dev/mapper/thin-pool

#### dm.directlvm_device

As an alternative to manually creating a thin pool as above, Docker can
automatically configure a block device for you.

Example use:

   $ dockerd \
         --storage-opt dm.directlvm_device=/dev/xvdf

##### dm.thinp_percent

Sets the percentage of passed in block device to use for storage.

###### Example:

   $ sudo dockerd \
        --storage-opt dm.thinp_percent=95

##### `dm.thinp_metapercent`

Sets the percentage of the passed in block device to use for metadata storage.

###### Example:

   $ sudo dockerd \
         --storage-opt dm.thinp_metapercent=1

##### dm.thinp_autoextend_threshold

Sets the value of the percentage of space used before `lvm` attempts to
autoextend the available space [100 = disabled]

###### Example:

   $ sudo dockerd \
         --storage-opt dm.thinp_autoextend_threshold=80

##### dm.thinp_autoextend_percent

Sets the value percentage value to increase the thin pool by when `lvm`
attempts to autoextend the available space [100 = disabled]

###### Example:

   $ sudo dockerd \
         --storage-opt dm.thinp_autoextend_percent=20

#### dm.basesize

Specifies the size to use when creating the base device, which limits
the size of images and containers. The default value is 10G. Note,
thin devices are inherently "sparse", so a 10G device which is mostly
empty doesn't use 10 GB of space on the pool. However, the filesystem
will use more space for base images the larger the device
is.

The base device size can be increased at daemon restart which will allow
all future images and containers (based on those new images) to be of the
new base device size.

Example use: `dockerd --storage-opt dm.basesize=50G`

This will increase the base device size to 50G. The Docker daemon will throw an
error if existing base device size is larger than 50G. A user can use
this option to expand the base device size however shrinking is not permitted.

This value affects the system-wide "base" empty filesystem that may already
be initialized and inherited by pulled images. Typically, a change to this
value requires additional steps to take effect:

        $ sudo service docker stop
        $ sudo rm -rf /var/lib/docker
        $ sudo service docker start

Example use: `dockerd --storage-opt dm.basesize=20G`

#### dm.fs

Specifies the filesystem type to use for the base device. The
supported options are `ext4` and `xfs`. The default is `ext4`.

Example use: `dockerd --storage-opt dm.fs=xfs`

#### dm.mkfsarg

Specifies extra mkfs arguments to be used when creating the base device.

Example use: `dockerd --storage-opt "dm.mkfsarg=-O ^has_journal"`

#### dm.mountopt

Specifies extra mount options used when mounting the thin devices.

Example use: `dockerd --storage-opt dm.mountopt=nodiscard`

#### dm.use_deferred_removal

Enables use of deferred device removal if `libdm` and the kernel driver
support the mechanism.

Deferred device removal means that if device is busy when devices are
being removed/deactivated, then a deferred removal is scheduled on
device. And devices automatically go away when last user of the device
exits.

For example, when a container exits, its associated thin device is removed. If
that device has leaked into some other mount namespace and can't be removed,
the container exit still succeeds and this option causes the system to schedule
the device for deferred removal. It does not wait in a loop trying to remove a
busy device.

Example use: `dockerd --storage-opt dm.use_deferred_removal=true`

#### dm.use_deferred_deletion

Enables use of deferred device deletion for thin pool devices. By default,
thin pool device deletion is synchronous. Before a container is deleted, the
Docker daemon removes any associated devices. If the storage driver can not
remove a device, the container deletion fails and daemon returns.

`Error deleting container: Error response from daemon: Cannot destroy container`

To avoid this failure, enable both deferred device deletion and deferred
device removal on the daemon.

`dockerd --storage-opt dm.use_deferred_deletion=true --storage-opt dm.use_deferred_removal=true`

With these two options enabled, if a device is busy when the driver is
deleting a container, the driver marks the device as deleted. Later, when the
device isn't in use, the driver deletes it.

In general it should be safe to enable this option by default. It will help
when unintentional leaking of mount point happens across multiple mount
namespaces.

#### dm.loopdatasize

**Note**: This option configures devicemapper loopback, which should not be
used in production.

Specifies the size to use when creating the loopback file for the "data" device
which is used for the thin pool. The default size is 100G. The file is sparse,
so it will not initially take up this much space.

Example use: `dockerd --storage-opt dm.loopdatasize=200G`

#### dm.loopmetadatasize

**Note**: This option configures devicemapper loopback, which should not be
used in production.

Specifies the size to use when creating the loopback file for the "metadata"
device which is used for the thin pool. The default size is 2G. The file is
sparse, so it will not initially take up this much space.

Example use: `dockerd --storage-opt dm.loopmetadatasize=4G`

#### dm.datadev

(Deprecated, use `dm.thinpooldev`)

Specifies a custom blockdevice to use for data for a Docker-managed thin pool.
It is better to use `dm.thinpooldev` - see the documentation for it above for
discussion of the advantages.

#### dm.metadatadev

(Deprecated, use `dm.thinpooldev`)

Specifies a custom blockdevice to use for metadata for a Docker-managed thin
pool.  See `dm.datadev` for why this is deprecated.

#### dm.blocksize

Specifies a custom blocksize to use for the thin pool.  The default
blocksize is 64K.

Example use: `dockerd --storage-opt dm.blocksize=512K`

#### dm.blkdiscard

Enables or disables the use of `blkdiscard` when removing devicemapper devices.
This is disabled by default due to the additional latency, but as a special
case with loopback devices it will be enabled, in order to re-sparsify the
loopback file on image/container removal.

Disabling this on loopback can lead to *much* faster container removal times,
but it also prevents the space used in `/var/lib/docker` directory from being
returned to the system for other use when containers are removed.

Example use: `dockerd --storage-opt dm.blkdiscard=false`

#### dm.override_udev_sync_check

By default, the devicemapper backend attempts to synchronize with the `udev`
device manager for the Linux kernel.  This option allows disabling that
synchronization, to continue even though the configuration may be buggy.

To view the `udev` sync support of a Docker daemon that is using the
`devicemapper` driver, run:

        $ docker info
        [...]
         Udev Sync Supported: true
        [...]

When `udev` sync support is `true`, then `devicemapper` and `udev` can
coordinate the activation and deactivation of devices for containers.

When `udev` sync support is `false`, a race condition occurs between the
`devicemapper` and `udev` during create and cleanup. The race condition results
in errors and failures. (For information on these failures, see
[docker#4036](https://github.com/docker/docker/issues/4036))

To allow the `docker` daemon to start, regardless of whether `udev` sync is
`false`, set `dm.override_udev_sync_check` to true:

        $ dockerd --storage-opt dm.override_udev_sync_check=true

When this value is `true`, the driver continues and simply warns you the errors
are happening.

**Note**: The ideal is to pursue a `docker` daemon and environment that does
support synchronizing with `udev`. For further discussion on this topic, see
[docker#4036](https://github.com/docker/docker/issues/4036).
Otherwise, set this flag for migrating existing Docker daemons to a daemon with
a supported environment.

#### dm.min_free_space

Specifies the min free space percent in a thin pool require for new device
creation to succeed. This check applies to both free data space as well
as free metadata space. Valid values are from 0% - 99%. Value 0% disables
free space checking logic. If user does not specify a value for this option,
the Engine uses a default value of 10%.

Whenever a new a thin pool device is created (during `docker pull` or during
container creation), the Engine checks if the minimum free space is available.
If the space is unavailable, then device creation fails and any relevant
`docker` operation fails.

To recover from this error, you must create more free space in the thin pool to
recover from the error. You can create free space by deleting some images and
containers from tge thin pool. You can also add more storage to the thin pool.

To add more space to an LVM (logical volume management) thin pool, just add
more storage to the  group container thin pool; this should automatically
resolve any errors. If your configuration uses loop devices, then stop the
Engine daemon, grow the size of loop files and restart the daemon to resolve
the issue.

Example use:: `dockerd --storage-opt dm.min_free_space=10%`

#### dm.xfs_nospace_max_retries

Specifies the maximum number of retries XFS should attempt to complete IO when
ENOSPC (no space) error is returned by underlying storage device.

By default XFS retries infinitely for IO to finish and this can result in
unkillable process. To change this behavior one can set xfs_nospace_max_retries
to say 0 and XFS will not retry IO after getting ENOSPC and will shutdown
filesystem.

Example use:

    $ sudo dockerd --storage-opt dm.xfs_nospace_max_retries=0

##### dm.libdm_log_level

Specifies the maxmimum libdm log level that will be forwarded to the dockerd
log (as specified by --log-level). This option is primarily intended for
debugging problems involving libdm. Using values other than the defaults may
cause false-positive warnings to be logged.

Values specified must fall within the range of valid libdm log levels. At the
time of writing, the following is the list of libdm log levels as well as their
corresponding levels when output by dockerd.

| libdm Level | Value | --log-level |
| ----------- | -----:| ----------- |
| _LOG_FATAL  |     2 | error       |
| _LOG_ERR    |     3 | error       |
| _LOG_WARN   |     4 | warn        |
| _LOG_NOTICE |     5 | info        |
| _LOG_INFO   |     6 | info        |
| _LOG_DEBUG  |     7 | debug       |

Example use:

    $ sudo dockerd \
	      --log-level debug \
          --storage-opt dm.libdm_log_level=7

## ZFS options

#### zfs.fsname

Set zfs filesystem under which docker will create its own datasets.  By default
docker will pick up the zfs filesystem where docker graph (`/var/lib/docker`)
is located.

Example use: `dockerd -s zfs --storage-opt zfs.fsname=zroot/docker`

## Btrfs options

#### btrfs.min_space

Specifies the minimum size to use when creating the subvolume which is used for
containers. If user uses disk quota for btrfs when creating or running a
container with **--storage-opt size** option, docker should ensure the **size**
cannot be smaller than **btrfs.min_space**.

Example use: `docker daemon -s btrfs --storage-opt btrfs.min_space=10G`

# CLUSTER STORE OPTIONS

The daemon uses libkv to advertise the node within the cluster.  Some Key/Value
backends support mutual TLS, and the client TLS settings used by the daemon can
be configured using the **--cluster-store-opt** flag, specifying the paths to
PEM encoded files.

#### kv.cacertfile

Specifies the path to a local file with PEM encoded CA certificates to trust

#### kv.certfile

Specifies the path to a local file with a PEM encoded certificate.  This
certificate is used as the client cert for communication with the Key/Value
store.

#### kv.keyfile

Specifies the path to a local file with a PEM encoded private key.  This
private key is used as the client key for communication with the Key/Value
store.

# Access authorization

Docker's access authorization can be extended by authorization plugins that
your organization can purchase or build themselves. You can install one or more
authorization plugins when you start the Docker `daemon` using the
`--authorization-plugin=PLUGIN_ID` option.

```bash
dockerd --authorization-plugin=plugin1 --authorization-plugin=plugin2,...
```

The `PLUGIN_ID` value is either the plugin's name or a path to its
specification file. The plugin's implementation determines whether you can
specify a name or path. Consult with your Docker administrator to get
information about the plugins available to you.

Once a plugin is installed, requests made to the `daemon` through the
command line or Docker's Engine API are allowed or denied by the plugin.
If you have multiple plugins installed, each plugin, in order, must
allow the request for it to complete.

For information about how to create an authorization plugin, see [access authorization
plugin](https://docs.docker.com/engine/extend/plugins_authorization/) section in the
Docker extend section of this documentation.

# RUNTIME EXECUTION OPTIONS

You can configure the runtime using options specified with the `--exec-opt` flag.
All the flag's options have the `native` prefix. A single `native.cgroupdriver`
option is available.

The `native.cgroupdriver` option specifies the management of the container's
cgroups. You can only specify `cgroupfs` or `systemd`. If you specify
`systemd` and it is not available, the system errors out. If you omit the
`native.cgroupdriver` option,` cgroupfs` is used on cgroup v1 hosts, `systemd`
is used on cgroup v2 hosts with systemd available.

This example sets the `cgroupdriver` to `systemd`:

```bash
$ sudo dockerd --exec-opt native.cgroupdriver=systemd
```

Setting this option applies to all containers the daemon launches.

# HISTORY
Sept 2015, Originally compiled by Shishir Mahajan <shishir.mahajan@redhat.com>
based on docker.com source material and internal work.