<!doctype debiandoc public "-//DebianDoc//DTD DebianDoc//EN" [
<!ENTITY svn-upgrade-docbookman SYSTEM "svn-upgrade.docbookman">
<!ENTITY svn-inject-docbookman SYSTEM  "svn-inject.docbookman">
<!ENTITY svn-buildpackage-docbookman SYSTEM  "svn-buildpackage.docbookman">
]>

<book>
<title>svn-buildpackage - maintaining Debian packages with Subversion</title>
<author>Eduard Bloch</author>
<version>
$LastChangedDate: 2005-09-23 16:17:33 +0200 (Fr, 23 Sep 2005) $
<!-- date-->
</version>

<copyright>
svn-buildpackage, all associated scripts and programs, this manual,
and all build scripts are Copyright &copy; 2003 Eduard Bloch.
<p>
See <ref id="copyright"> for details.
</copyright>

<toc sect>

<chapt id="intro">Introduction

<sect id="pur">Purpose
<p>
This short document is only intended to give a short help in
converting packages to Subversion management. It is primarily intended
for developers not really familiar with Subversion or CVS management
and/or converting from maintaining their packages using common tools
(dpkg-dev, devscripts) only to version control system Subversion.
</sect>

<sect id="why">Why a version control system?
<p>
But the first question may be: why use a version control system at all?
Look at how the source is handled by the Debian package. First, we have
the pure upstream source, which is often maintained by another person.
The upstream author has his own development line and releases the source
in snapshots (often called releases or program versions).
<p>
The Debian maintainer adds an own set of modifications, leading to an
own version of the upstream package. The difference set between this two
version finally ends in Debian's .diff.gz files, and this patchset is
often appliable to future upstream versions in order to get the "Debian
versions".
<p>
So the obvious way to deal with source upgrades/changes is using local
copies, patch, different patchutils and scripts to automate all this,
eg. uupdate. However, it often becomes nasty and uncomfortable, and
there is no way to undo changes that you may do by mistakes.
<p>
At this point, the Subversion system can be used to simplify that work.
It does the same things that you normaly would do by-hand but keeps it
in an own archive (a repository). It stores the development lines of
Upstream and Debian source, keeping them in different directories
(different branches). The branches are wired internally (the VCS "knows"
the history of the file and tracks the differences between the Upstream
and Debian versions). When a new upstream version is installed, the
differences between the old and new upstream versions and the Debian
version are merged together.
<p>
You can create snapshots of your Debian version ("tag" it) and switch
back to a previous state, or see the changes done in the files. You can
store when commiting the file to the repository or place custom tags on
the files ("properties") serving various purposes.
</sect>

<sect id="feat">Features
<p>
  svn-buildpackage and other scripts around it has been created to do
  follow things:
<p>
<list>
  <item>
    keep Debian package under revision control, which means storing
    different versions of files in a Subversion repository
  <item>
   allow easy walking back trough time using svn command
  <item>
   easy retrieval of past versions
  <item>
   keep track of upstream source versions and modified Debian versions
  <item>
   easy installation of new upstream versions, merging the Debian
   changes into it when needed (similar to the uupdate program)
  <item>
   automated package building in clean environment, notifying about
   uncommited changes
  <item>create version tags when requested to do the final build and
   update changelog when needed
  <item>allow co-work of multiple Debian developers on the same project
  <item>auto-configure the repository layout, making it easy to use by
  people without knowing much about Subversion usage (mostly you need only the
  add, rm and mv commands of svn)
  <item>allow to store only the Debian specific changes in the
  repository and merge them into the upstream source in the build area
  (which nicely completes build systems like dpatch or dbs)
  <item>If wished, keep the upstream tarballs inside of the repository
</list>
</sect>

<sect id="over">Contents overview
<p>
There are currently three scripts provided by the svn-buildpackage
package: 
<list>
<item>
svn-inject: script used to insert an existing Debian package into a
Subversion repository, creating the repository layout as needed.
<item>svn-buildpackage: exports the contents of the directory associated
with the starting directory from the Subversion repository to the clean
environment and build the package there
<item>svn-upgrade: similar to uupdate, upgrades the trunk to a new
upstream version, preserving and merging Debian specific changes
</list>
</sect>
<sect>Popular repository layouts
<p>
There are different ways to store the packages in the repositories (or
in multiple repositories at your choice). <tt>svn-buildpackage</tt>
normaly expects a directory structure similar to the one well described in the
Subversion Book, which looks like:
<example>
packageA/
   trunk/
   branches/
   branches/upstream
   tags/

projectB/
   trunk/
   branches/
   branches/developerFoo
   tags/
</example>
packageA above may be a typical upstream-based source package and a
projectB may be a Debian native package with a separate branch created
by developerFoo for his own experiments. See <url name="Subversion
Book/Branches"
id="http://svnbook.red-bean.com/html-chunk/ch04s02.html"> for more
details about using Subversion branches.
<p>
Also note that Tags work quite different than those in CVS. Subversion
does not maintain magic tags associated with some files. Instead, it
tracks the file state and moves, so Tagging something means creating a
copy (inside of the Repository, harddisk-space efficient) of a certain
version of the file set. So the Debian branch of the released package
source is contained in <tt>trunk/</tt> and is tagged by copying
(mirroring) the trunk tree to <tt>tags/DEBIAN-REVISION</tt>. The same
happens for the upstream releases. In addition, the most recent upstream
version is mirrored to <tt>branches/upstream/current</tt>. After few
package upgrade cycles, the directory tree may look so:
<example>
# svn ls -R file:///home/user/svn-repo/dev/translucency 
branches/
branches/upstream/
branches/upstream/0.5.9/
branches/upstream/0.5.9/AUTHORS
branches/upstream/0.5.9/COPYING
...
branches/upstream/0.6.0/
branches/upstream/0.6.0/AUTHORS
branches/upstream/0.6.0/COPYING
...
branches/upstream/current/
branches/upstream/current/AUTHORS
branches/upstream/current/COPYING
... same stuff as in 0.6.0 ...
tags/
tags/0.5.9-1/
...
tags/0.5.9-1/debian/
tags/0.5.9-1/debian/README.Debian
...
tags/0.6.0-1/
tags/0.6.0-1/AUTHORS
...
tags/0.6.0-1/debian/
tags/0.6.0-1/debian/README.Debian
tags/0.6.0-1/debian/changelog
...
trunk/
trunk/AUTHORS
trunk/COPYING
... trunk where 0.6.0-2 is beeing prepared ...
</example>
<p>
svn-buildpackage also supports the second repository layout suggested in
the Subversion Book (function/package) but svn-inject prefers the one
documented above. Both svn-buildpackage and svn-upgrade should be able to
auto-detect the repository layout and the location of package files.
<p>
In theory, you do not have to follow that examples and place the trunk,
branches and tags directory on the locations you like more.
But svn-buildpackage and other scripts won't locate the files
automaticaly so you will need to edit the .svn/deb-layout file in your
working directory and set paths. See the old <url name="abstract"
id="file:///usr/share/doc/svn-buildpackage/CONFIG"> about how
auto-detection works and the <url name="config example"
id="file:///usr/share/doc/svn-buildpackage/examples/config.example">.
<p>
Finaly, the working directory structure on your development system may
look so:
<example>
dev/ # base directory, may be under version control or not
dev/foo # trunk directories of various packages
dev/bar # contents correspond to trunk, see above
dev/tarballs # where "orig" tarballs are stored, may be under VC or not
dev/build-area # where the packages are exported temporarily and built
</example>
</sect>
</chapt>
	
<chapt id="start">Getting started
<p>
Besides of the packages that are installed by dependencies when you
install <tt>svn-buildpackage</tt>, you may need <tt>ssh</tt> and the
obligatory tool chain: <tt>dpkg-dev</tt>, <tt>build-essential</tt> and
all the packages they pull into the system.

<sect id="quick">Quick guide
<p>
Here is a quick guide for those who wish to build an existing package
using an existing, public available SVN repository. To create own
repositories, skip this section and look for more details below.
<list>
<item>
svn co &lt;svn://server/path/to/trunk&gt; package
<item>
mkdir tarballs
<item>
cp dir-where-you-keep-the-source/package_version.orig.tar.gz tarballs/
<p>
NOTE: you need the upstream source tarballs, stored under a usual dpkg-source-compatible filename in tarballs/
<item>
cd package
<item>
svn-buildpackage -us -uc -rfakeroot
</list>
</sect>
<sect id="basics">Basic svn usage
<p>
You need only few commands to start using svn with svn-buildpackage
scripts. If you wish to learn more about it, read parts of the the 
<url name="Subversion Book"
id="http://svnbook.red-bean.com/html-chunk/">. The most used commands
are:
<list>
<item>
<tt>add</tt> -- put new files unto the revision contol
<item>
<tt>rm</tt> -- remove the files from the repository
<item>
<tt>mv</tt> -- move files around, leting revision control system know about it
<item>
<tt>commit</tt> -- commit your changes to the repository
<item>
<tt>resolved</tt> -- tell svn that you have resolved a conflict
<item>
<tt>diff</tt> -- creates a "diff -u" between two versions, specified by
file revision number or by date. See the <tt>diff --help</tt> output.
<item>
<tt>cat -r Revision</tt> -- useful to browse in some previous revision
of the file
</list>
If you are familiar with CVS you will probably know almost all you need.
</sect>

<sect id="svncreate">Creating Subversion repository
<p>
The main Subversion repository is easily created with:
<example>
svnadmin create repo-directory
</example>
For our example, we choose the name <tt>svn-deb-repo</tt> and put it in
<tt>/home/user</>. 
<p>
If you plan to keep many packages in the one
repository including upstream tarballs, consider to put it on a hard
disk with much free space and good performance (especially short disk
access times) since the repository will grow and the filesystem may
become fragmented over time.
</sect>


<sect id="multidev">Using by multiple developers
<p>
Multiple developers with local access to the repository may share it
using a common group. To do so, create a new group and add all
developers to it. Run "chgrp -R sharedGroup repdir ; chmod -R g+s
repdir" for the shared group and the repository directory. Now,
on local access to this repository everybody will create files with the
appropriate group setting. However, the developers will need to set a
liberal umask before using svn (like "0022").
<p>
If somebody resists to do
so, there is still a brute-force solution: fix the permissions with a
post-commit script. However, this is an "unsound" solution and may lead
to ALL KINDS OF PROBLEMS. MAKE SURE THAT YOU ARE AWARE OF THE POSSIBLE
CONSEQUENCES BEFORE YOU OPEN THE PANDORA BOX. See <url
id="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=240630"
name="Debian BTS"> for details. When you damage your repository, don't
blame me and remember that there is "svnadmin recover".
<p>
<example>
#!/bin/sh

# POST-COMMIT HOOK
# The following corrects the permissions of the repository files

REPOS="$1"
REV="$2"

chgrp -R sharedGroup $REPOS
# replace sharedGroup with your group
chmod -R g+r $REPOS
chmod -R g+w $REPOS
</example>

<sect1 id="svn-d-o">SVN over SSH
<p>
To run Subversion over SSH, you basically need a shell on the target
system and a subversion repository located there which is created
following the description above. The repository must be configured for
access by the system users of the remote system.
<p>
Assuming that your user name on the client system is the same as on the
server side, there is not much to configure. Just change the protocol
specificaton from <tt>file://</tt> to
<tt>svn+ssh://remoteusername@server-hostname</tt> in all examples showed
in this manual.
<p>
Note that during svn-buildpackage tools actions a lot of SSH calls can be made
and so the user is asked for his login data. The regular method to deal with
that is using an SSH key authentication method together with ssh-agent and
ssh-add to cache the passphrase in memory. Another approach, which also brings
a significant speed boost, is using a cached SSH connection. This can be done
with a new feature of OpenSSH (see <url name="GCC SSH connection caching howto" id="http://gcc.gnu.org/wiki/SSH%20connection%20caching">) or a third-party tool like <tt>fsh</tt>.
<p>
If you wish to use fsh over ssh you could specify a custom transport method in
subversions's configuration. To do so, edit the file
<tt>~/.subversion/config</tt> and add the section <tt>[tunnels]</tt> to it,
following by your custom transport definition.
Example:
<p>
<example>
# personal subversion config with custom ssh tunnel command
[tunnels]
# SSH account on svn.d.o
# compression is enabled in the ssh config
deb = fsh -l blade
# SSH account for NQ intranet, set fix username
nq = ssh -C -l zomb
</example>
You can use the new defined tunnels in a similar ways as described above
but replace <tt>svn+ssh</tt> with <tt>svn+tunnelname</tt>, so the final
URL looks like:
<example>svn+deb://svn.debian.org/svn/myproject/ourpackage/trunk</example>.
</sect1>
<sect1 id="anon">Anonymous access
<p>
You can allow outsiders to have anonymous (read-only) access using the
svnserve program, as described in the Subversion documentation.
<p>
Another method is using HTTP/WebDAV with Apache2. More about a such
setup can be found in the Subversion Book and the <url name="SubversionApache2SSL Howto" id="http://wiki.debian.net/?SubversionApache2SSLHowto">. <url
id="http://svn.debian.org/" name="svn.debian.org"> is an example site
granting anonymous access to some selected projects hosted there.
</sect1>
</sect>
</chapt>

<chapt id="import">Importing Debian packages
<p>
<sect>Importing from existing source package files
<p>
The <tt>svn-inject</tt> utility is intended to import already packaged source
packages into a new subdirectory of the repository, creating the
repository layout as needed. Normally, it takes two arguments: the
.dsc file of your package and the base URL of the Subversion
repository.

<example>
svn-inject translucency_*dsc file:///tmp/z
cp /tmp/translucency_0.6.0.orig.tar.gz /tmp/tarballs || true
mkdir -p translucency/branches/upstream
tar  -z -x -f /tmp/translucency_0.6.0.orig.tar.gz
mv * current
svn -q import -m"Installing original source version" translucency file:///tmp/z/translucency
svn -m Tagging upstream source version copy file:///tmp/z/translucency/branches/
upstream/current file:///tmp/z/translucency/branches/upstream/0.6.0 -q
svn -m Forking to Trunk copy file:///tmp/z/translucency/branches/upstream/current file:///tmp/z/translucency/trunk -q
dpkg-source -x /tmp/translucency_0.6.0-1.dsc
dpkg-source: extracting translucency in translucency-0.6.0
svn_load_dirs file:///tmp/z/translucency/trunk . *
...
Running /usr/bin/svn propset svn:executable  initscript
Running /usr/bin/svn propset svn:executable  debian/rules
Running /usr/bin/svn propset svn:executable  mounttest.sh
Running /usr/bin/svn propset svn:executable  mount.translucency
Running /usr/bin/svn propget svn:eol-style base.h
Running /usr/bin/svn propget svn:eol-style Makefile
Running /usr/bin/svn propget svn:eol-style translucency.8
Running /usr/bin/svn commit -m Load translucency-0.6.0 into translucency/trunk.

Running /usr/bin/svn update
Cleaning up /tmp/svn_load_dirs_jD7OenzVjI
Storing trunk copy in /tmp/translucency.
svn co file:///tmp/z/translucency/trunk /tmp/translucency -q
svn propset svn:executable 1 debian/rules -q
svn -m"Fixing debian/rules permissions" commit debian -q
Done! Removing tempdir.
Your working directory is /tmp/translucency - have fun!
</example>
<p>
If you omit the URL, <tt>svn-inject</tt> will try to use the URL of the
current directory as base URL. I would not rely on this, however.
</sect>

<sect>On-Build-Time merging
<p>
A special feature of svn-buildpackage is so called
mergeWithUpstream-mode. Many projects do not want to keep the whole upstream source under
revision control, eg. because of the large amount of required disc space
and process time. Sometimes it makes sense to keep only the
<tt>debian/</tt> directory any maybe few other files under revision
control.
<p>
The task of exporting the source from repository and adding
it to the upstream source before building becomes annoying the time. But
the <tt>svn-buildpackage</tt> tools automate most of this work for you:
they switch to so called mergeWithUpstream-mode if a special flag has
been detected: the mergeWithUpstream (Subversion) property of the
<tt>debian</tt> directory. <tt>svn-buildpackage</tt> will merge the trunk with
upstream source on build time and <tt>svn-upgrade</tt> will only update
the changed files in this case.
<p>
To enable this feature during the inital import of the source package,
simply add the <tt>-o</tt> switch to the <tt>svn-inject</tt> call and it
will prepare the source for with mergeWithUpstream-mode: reduce
the set of files to those modified for Debian and set the
mergeWithUpstream property.
<p>
But what, if you decide to switch to mergeWithUpstream-mode after the
package has been injected? To do this, checkout the whole repository,
remove the files not changed in the Debian package from both upstream
source and Debian branch (<tt>svn rm</tt>) and set the mergeWithUpstream property on <tt>debian</tt> in the
<tt>trunk</tt> directory with <tt>svn propset mergeWithUpstream 1 debian</tt>.
<p>
If you actually decide to stop using the mergeWithUpstream-mode, unset the
mergeWithUpstream property as follows: <tt>svn propdel mergeWithUpstream debian/</tt>.
<p>
If you don't want to store the upstream sources of all your packages in the
repository, you can pass the <tt>--no-branches</tt> switch to
<tt>svn-inject</tt>, which will prevent <tt>svn-inject</tt> from creating a
<tt>branches</tt> subdirectory.
</sect>

<!--sect>Importing directories
<p>
Importing directories is a special case and I do not recommend using
this feature. The implementation may be buggy and wreak random havoc.
You have been warned!
<p>
You can specify source directories as input for svn-inject and
svn-upgrade. This may be useful for two things:

<list>
   <item>
   Inserting pristine upstream source into the repository before running
   dh_make. You may wish to do so if you want to use Subversion from the
   beginning. If the version string cannot be extracted from the directory
   name, you will have to specify it manually with the
   <strong>-V</strong> option.
   <item>tracking a local version of a Debian package with custom
   changes if you are not the maintainer. So you can inject and upgrade
   from the debianized package source (extracted with "dpkg-source -x")
   but keep your personal changes in the <tt>trunk</tt>.
</list>
</sect-->
</chapt>

<chapt id="tasks">Common tasks
<p>
<sect id="checkout">Checkout
<p>
svn-inject will do the initial checkout for you. If you need another
working copy, run
<example>
svn co protocol://repository-base-url/yourpackage
</example>
</sect>
<sect id="build">Building the package
<p>
Change to your trunk directory and run:
<example>
svn-buildpackage -us -uc -rfakeroot
</example>
You may recognice the options above -- they are passed directly to the
build command (<tt>dpkg-buildpackage</tt> by default). Normaly, the
build is done in another directory (exporting the source with cp-la-like
method). If you wish the resulting packages to be placed in the
directory above, use the <tt>--svn-move</tt> option. To run Lintian
after the build, use <tt>--svn-lintian</tt> option. More options are
described in the manpage (<ref id="sbman">).
</sect>
<sect id="handlesoure">Working with source
<p>
Every time when you add or modify something, <tt>svn-buildpackage</tt>
won't let you proceed unless suspicious files are in the clean state
(unless you use the <tt>--svn-ignore</tt> switch).  You use the commands
described in <ref id="basics"> to register the new files (or move or
delete the old ones) and commit the changes to the repository.
</sect>
<sect>Handling new upstream versions
<p>
Upgrading with new upstream version normaly happens in two steps:
<list>
<item>
the <tt>current</tt> tree in the upstream branch is upgraded with the
source from the new upstream package (the old version is kept in
repository in <tt>branches/upstream/oldVersion</tt>).
<item>
The version in <tt>trunk/</tt> becomes upgraded by merging the changes
between the upstream versions into the <tt>trunk/</tt> directory.
</list>
The script <tt>svn-upgrade</tt> (formerly <tt>svn-uupdate</tt>) does
both things for you and also creates a new changelog entry. The first step
is done internaly by using a third party script (svn_load_dirs, see Subversion
book for documentation), the second step is done with the merge command
of svn. Just run <tt>svn-upgrade</tt> from you local working directory
(which corresponds the <tt>trunk/</tt> checkout product).
<p>
After running <tt>svn-upgrade</tt> some files may be in conflicting
state. This is naturaly happens if you have modified some files in the
upstream package and now upstream did something similar on the same
positions so <tt>svn merge</tt> was confused.
<p>
When <tt>svn-upgrade</tt> complains about files in conflicting state,
fix them manualy. When done, use the <tt>svn resolved</tt> command to
mark them as clean and <tt>svn commit</tt> to update the repository.
</sect>
<sect>Finalizing the Revision
<p>
When you are ready to upload a new revision of your package, everything
builds fine, the changelog is cleaned up and the package is tested, you
can do the final build and tag the end version. To do so, add
<tt>--svn-tag</tt> switch and after the package is built, it will be
tagged (by creating a copy of the <tt>trunk/</tt> directory as said above).
</sect>
</chapt>

<chapt id="reference">Command reference
<p>
<sect>svn-inject
<p>
&svn-inject-docbookman;
</sect>

<sect id="sbman">svn-buildpackage
<p>
&svn-buildpackage-docbookman;

</sect>

<sect>svn-upgrade
<p>
&svn-upgrade-docbookman;
</sect>

</chapt>

<chapt id="misc">Further documentation
<sect>Various links
<p>
<list>
<item>
Subversion Homepage: <url id="http://subversion.tigris.org/"
name="http://subversion.tigris.org/">
<item>
The Subversion Book: <url 
id="http://svnbook.red-bean.com/" 
name="http://svnbook.red-bean.com/">
<item>
Subversion vs. CVS and others: <url name="http://better-scm.berlios.de/"
id="http://better-scm.berlios.de/">
</list>
</sect>
<sect id="copyright">Copyright
<p>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
<p>
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
<p>
A copy of the GNU General Public License is available as <url
id="file:///usr/share/common-licenses/GPL"
name="/usr/share/common-licenses/GPL"> in the Debian GNU/Linux distribution or on the World Wide Web at 
<url name="http://www.gnu.org/copyleft/gpl.html"
id="http://www.gnu.org/copyleft/gpl.html">. You can also obtain it by
writing to the Free Software Foundation, Inc., 59 Temple Place, Suite
330, Boston, MA 02111-1307, USA.
</sect>
</chapt>
</book>