1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
|
# Running tilemaker
You can get a run-down of available options with
tilemaker --help
The following document explains each option.
## Standard usage
To create vector tiles from an OpenStreetMap .pbf extract, tilemaker's standard syntax is:
tilemaker --input oxfordshire.osm.pbf \
--output oxfordshire.mbtiles \
--config resources/config-openmaptiles.json \
--process resources/process-openmaptiles.lua
The `--config` and `--process` arguments are the paths of your JSON config file and Lua
processing script. These are described in CONFIGURATION.md. Here we're using the ready-made
OpenMapTiles-compatible script.
You can output to an .mbtiles or .pmtiles file. mbtiles is widely supported and easy to serve
(it's an sqlite database under the hood). [pmtiles](https://github.com/protomaps/PMTiles) is
a newer format optimised for serving over the cloud. You can also write tiles directly to the
filesystem by specifying a directory path for `--output`.
This is all you need to know, but if you want to reduce memory requirements, read on.
## Using on-disk storage
By default, tilemaker uses RAM to store the nodes and ways it reads from the .osm.pbf, prior
to writing them out as vector tiles. This is fine for small regions, but can impose high memory
requirements for larger areas.
To use on-disk storage instead, pass the `--store` argument with a path to the directory where
you want the temporary store to be created - on an SSD or other fast disk. This allows you
to process bigger areas even on memory-constrained systems.
## Performance and memory tuning
By default, tilemaker aims to balance memory usage with speed, but with a slight tilt towards
minimising memory. You can get faster runtimes, at the expense of a little more memory, by
specifying `--fast`.
This is all most people will need to know. But if you have plentiful RAM, you can experiment
with these options. In general, options that use more memory run faster - but if you can
optimise memory such that your dataset fits entirely in RAM, this will be a big speed-up.
(`--fast` simply chooses a set of these options for you.)
* `--compact`: Use a smaller, faster data structure for node lookups. __Note__: This requires
the .pbf to have nodes in sequential order, typically by using `osmium renumber`.
* `--no-compress-nodes` and `--no-compress-ways`: Turn off node/way compression. Increases
RAM usage but runs faster.
* `--materialize-geometries`: Generate geometries in advance when reading .pbf. Increases RAM
usage but runs faster.
* `--shard-stores`: Group temporary storage by area. Reduces RAM usage on large files (e.g.
whole planet) but runs slower.
You can also tell tilemaker to only look at .pbf objects with certain tags. If you're making a
thematic map, this allows tilemaker to skip data it won't need. Specify this in your Lua file
like one of these three examples:
-- Only include major roads
way_keys = {"highway=motorway", "highway=trunk", "highway=primary", "highway=secondary"}`
-- Only include railways
way_keys = {"railway"}
-- Include everything but not buildings
way_keys = {"~building"}
## Merging
You can specify multiple .pbf files on the command line, and tilemaker will read them all in
before writing the vector tiles.
Alternatively, you can use the `--merge` switch to add to an existing .mbtiles. Create your
.mbtiles in the usual way:
tilemaker --input australia.osm.pbf \
--output oceania.mbtiles \
[...]
Then rerun with another .pbf, using the `--merge` flag:
tilemaker --input new-zealand.osm.pbf \
--output oceania.mbtiles \
--merge \
[...]
The second run will proceed a little more slowly due to reading in existing tiles in areas which
overlap. Any OSM objects which appear in both files will be written twice.
If you're very pushed for memory, you could potentially use `osmium tags-filter` to split a
.pbf into several "thematic" extracts: for example, one containing buildings, another roads,
and another landuse. Renumber each one, then run tilemaker several times with `--merge` to add
one theme at a time.
## Output messages
Running tilemaker with the `--verbose` argument will output any issues encountered during tile
creation.
You may see geometry errors reported by Boost::Geometry. This typically reflects an error
in the OSM source data (for example, a multipolygon with several inner rings but no outer ring).
Often, if the geometry could not be written to the layer, the error will subsequently show in
a failed attempt to add attributes afterwards.
If you see a (possibly fatal) error about nodes missing from ways, or ways missing from
relations, this suggests your source .osm.pbf is malformed. This will often happen if you have
used another program to clip the .osm.pbf with a bounding polygon. You can tell tilemaker to
ignore missing nodes in ways with `--skip-integrity`, but it can't fix missing ways in
multipolygon relations. Instead, tell your clipping utility to create a well-formed file using
`--strategy=smart` (Osmium) or `clipIncompleteEntities=true` (Osmosis).
tilemaker is meant for use with OSM data. It will likely not work if you add your own data
to the .osm.pbf file with unusual IDs (e.g. negative IDs or very large numbers). If you must
do this, use a tool like `osmium renumber` first to get the IDs back to a normal range.
## Github Action
You can integrate tilemaker as a Github Action into your [Github Workflow](https://help.github.com/en/actions).
Here is an example:
```yaml
- uses: systemed/tilemaker@v2.0.0
with:
# Required, same to --input
input: /path/to/osm.pbf
# Required, same to --output. Could be a directory or a .mbtiles files
output: /path/to/output
# Optional, same to --config
# If not being set, default to resources/config-openmaptiles.config
config: /path/to/config
# Optional, same to --process
# If not being set, default to resources/process-openmaptiles.lua
process: /path/to/lua
# Optional, other arguments
# If not being set, default to '--verbose'
extra: --threads 0
```
|
