osm2pgsql
=========
Converts OSM planet.osm data to SQL suitable for loading into 
a PostgreSQL database and then rendered into tiles by Mapnik.

The format of the database is optimised for ease of rendering
by mapnik. It may be less suitable for other general purpose
processing.

For a broader view of the whole map rendering tool chain see
http://wiki.openstreetmap.org/index.php/Mapnik and
http://wiki.openstreetmap.org/index.php/Slippy_Map

Any questions should be directed at the osm dev list
http://wiki.openstreetmap.org/index.php/Mailing_lists 

Changes
=======
The current version of code features several enhancements
and changes over the previous code:

- A direct connection to the Postgres database is used
  instead of outputting SQL
- Incorporation of setup_z_order.sql script so that
  post-processing is no longer required
- Able to read .gz and .bz2 files directly
- Performs UTF8Sanitize while reading the planet.osm
- Simplified usage: "osm2pgsql planet-xxxxxx.osm.bz2"
- Geometries are now stored in mercator projection
- Introduction of databased backed middle-layer for
  systems with low RAM (normally disabled).
- Support the choice of a few different projections
- Configurable table names

Requirements
============
The code is written in C and C++ and relies on the libraries
below:
- libxml2    http://xmlsoft.org/
- geos       http://geos.refractions.net/
- proj       http://www.remotesensing.org/proj/
- bzip2      http://www.bzip.org/
- zlib       http://www.zlib.net/
- PostgreSQL http://www.postgresql.org/
- PostGIS    http://postgis.refractions.net/

To make use of the database generated by this tool you will
probably also want to install:
- Mapnik from http://mapnik.org/

Building
========
On most Unix-like systems the program can be compiled by
running 'make'.

Note: the database connection parameters are hard coded to
connect to Postgres on localhost with the current user to
the database 'gis'.

Operation
=========

First you must have setup Postgres with a database named 'gis'
as per http://wiki.openstreetmap.org/index.php/Mapnik

1) Connects to database and creates the following 4 tables:
   - planet_osm_point
   - planet_osm_line
   - planet_osm_roads
   - planet_osm_polygon
The prefix "planet_osm" can be changed with the --prefix option, 
the above is the default.

2) Runs an XML parser on the input file (typically planet.osm) 
 and processes the nodes, segments and ways.

3) If a node has a tag declaring one of the attributes below then
 it is added to planet_osm_point. If it has no such tag then
 the position is noted, but not added to the SQL.

	name, place, landuse, waterway, highway, 
	railway, amenity, tourism, learning	

4) Segments are not output in the XML, they are used purely to 
 locate the nodes during way processing.

5) Ways are read in and the segments are examined to determine
 contiguous sequences by WKT(). Each sequence is added to
 the tables. If way consists of several dis-joint sequences of
 segments then multiple lines will be generated with the  
 osm_id of the original way.

6) Ways with the tags landuse or leisure are added to the
 planet_osm_polygon table. Other ways are added to
 planet_osm_line. Roads are also added to planet_osm_roads

7) Indexes are added to speed up the queries by Mapnik.

A quick note on projections
===========================

Depending on the command-line switches you can select which projection you
want the database in. You have three choices:

3395: The WGS84 mercator projection, used in the tile output
4326: The standard lat/long coordinates
900913: The spherical mercator projection, used by TileCache, Google Earth etc.

Depending on what you're using one or the other is appropriate. Most of the
current Mapnik tools and style sheets are configured for 3395 and reproject
to 900913 on the fly. But if you like you can project it correctly in one
step, but don't forget to change the Mapnik config to match.

Combining the -v and -h switches will tell about the exact definitions of
the projections.

In case you want to use some completely different projection there is the -E
option. It will initialise the projection as +init=epsg:<num>. This allows
you to use any projection recognised by proj4, which is useful if you want
to make a map in a different projection. These projections are usually
defined in /usr/share/proj/epsg.

Database Access Examples
========================
If you wish to access the data from the database then the
queries below should give you some hints:

$ psql gis
gis=> \d
               List of relations
 Schema |        Name        | Type  |  Owner
--------+--------------------+-------+----------
...
 public | planet_osm_line    | table | jburgess
 public | planet_osm_point   | table | jburgess
 public | planet_osm_polygon | table | jburgess
 public | planet_osm_roads   | table | jburgess
...

gis=> \d planet_osm_line
  Table "public.planet_osm_line"
  Column   |   Type   | Modifiers
-----------+----------+-----------
 osm_id    | integer  |
 name      | text     |
 place     | text     |
 landuse   | text     |
... [ lots of stuff deleted ] ...
 way       | geometry | not null
 z_order   | integer  | default 0


Each of the tables contains a subset of the planet.osm file representing
a particular geometry type
- Point contains nodes which have interesting tags
  e.g. place=city, name=London
  
- Line contains ways with interesting tags
  e.g. highway=motorway, ref=M25
  
- Polygon contains ways which form an enclosed area
  e.g. landuse=reservoir

The DB columns are used as follows:
- osm_id = the planet.osm ID of the node(point) or way(line,polygon)
- name, place, landuse, ... = the value of the given key, if present on
the node/way. If the tag is not present, the value is NULL. Only a
subset of all possible tags are stored in the DB. Only ones rendered in
the osm.xml are actually interesting to mapnik.
- way = PostGIS geometry describing the physical layout of the object.


Querying specific data requires knowlege of SQL and the OSM key/value
system, e.g.

gis=> select osm_id,astext(way),name from planet_osm_point where amenity='cinema' limit 5;
  osm_id  |                  astext                   |        name
----------+-------------------------------------------+--------------------
 26236284 | POINT(-79.7160836579093 43.6802306464618) |
 26206699 | POINT(51.4051989797638 35.7066045032235)  | Cinema Felestin
 26206700 | POINT(51.3994885141459 35.7058460359352)  | Cinema Asr-e Jadid
 20979630 | POINT(151.225781789807 -33.8943079539886) | Paris Cinema
 20979684 | POINT(151.226855394904 -33.8946830511095) | Hoyts
(5 rows)

Mapnik renders the data in each table by applying the rules in the
osm.xml file.


> How could I get e.g. all highways in a given bounding box?

The 'way' column contains the geo info and is the one which you need to
use in your WHERE clause. e.g.

gis=> select osm_id,highway,name from planet_osm_line where highway is not null and way && GeomFromText('POLYGON((0 52, 0.1 52, 0.1 52.1, 0 52.1, 0 52))',4326);
 
osm_id  |   highway    |       name
---------+--------------+------------------
 4273848 | unclassified |
 3977133 | trunk        | to Royston (tbc)
 4004841 | trunk        |
 4019198 | trunk        |
 4019199 | trunk        |
 4238966 | unclassified |


See the Postgis docs for details, e.g.
http://postgis.refractions.net/docs/ch04.html