<chapter id="build"><title>Building the package</title>
<para>
The rewrite of this tutorial document with updated contents and more practical examples is available as <ulink url="&guidedeb;">Guide for Debian Maintainers</ulink>.  Please use this new tutorial as the primary tutorial document.
</para>
<para>
We should now be ready to build the package.
</para>
<section id="completebuild"><title>Complete (re)build</title>
<para>
In order to perform a complete (re)build of a package properly, you
need to make sure you have installed
</para>
<itemizedlist>
<listitem>
<para>
the <systemitem role="package">build-essential</systemitem> package,
</para>
</listitem>
<listitem>
<para>
packages listed in the <literal>Build-Depends</literal> field (see <xref linkend="control"/>), and
</para>
</listitem>
<listitem>
<para>
packages listed in the <literal>Build-Depends-indep</literal> field (see <xref linkend="control"/>).
</para>
</listitem>
</itemizedlist>
<para>
Then you issue the following command in the source directory:
</para>
<screen>
$ dpkg-buildpackage -us -uc
</screen>
<para>
This will do everything to make full binary and source packages for you.  It
will:
</para>
<itemizedlist>
<listitem>
<para>
clean the source tree (<literal>debian/rules clean</literal>)
</para>
</listitem>
<listitem>
<para>
build the source package (<literal>dpkg-source -b</literal>)
</para>
</listitem>
<listitem>
<para>
build the program (<literal>debian/rules build</literal>)
</para>
</listitem>
<listitem>
<para>
build binary packages (<literal>fakeroot debian/rules binary</literal>)
</para>
</listitem>
<listitem>
<para>
make the <filename>.dsc</filename> file
</para>
</listitem>
<listitem>
<para>
make the <filename>.changes</filename> file, using
<command>dpkg-genchanges</command>
</para>
</listitem>
</itemizedlist>
<para>
If the build result is satisfactory, sign the <filename>.dsc</filename> and
<filename>.changes</filename> files with your private GPG key using the
<command>debsign</command>  command.  You need to enter your secret pass
phrase, twice.
<footnote><para>
This GPG key must be signed by a Debian developer to get connected to the web
of trust and must be registered to <ulink url="&keyring;">the Debian
keyring</ulink>.  This enables your uploaded packages to be accepted to the
Debian archives.  See 
<ulink url="&keycreate;">Creating a new GPG key</ulink> and
<ulink url="&keysigning; ">Debian Wiki on Keysigning</ulink>.
</para></footnote>
</para>
<para>
For a non-native Debian package, e.g.,
<systemitem role="package">gentoo</systemitem>, you will see the following
files in the parent directory (<filename>~/gentoo</filename>) after building
packages:
</para>
<itemizedlist>
<listitem>
<para>
<filename>gentoo_0.9.12.orig.tar.gz</filename>
</para>
<para>
This is the original upstream source code tarball, merely renamed to the above so that
it adheres to the Debian standard.  Note that this was created initially by the command
<literal>dh_make -f ../gentoo-0.9.12.tar.gz</literal>.
</para>
</listitem>
<listitem>
<para>
<filename>gentoo_0.9.12-1.dsc</filename>
</para>
<para>
This is a summary of the contents of the source code.  The file is generated
from your <filename>control</filename> file, and is used when unpacking the
source with <citerefentry> <refentrytitle>dpkg-source</refentrytitle>
<manvolnum>1</manvolnum> </citerefentry>.
</para>
</listitem>
<listitem>
<para>
<filename>gentoo_0.9.12-1.debian.tar.gz</filename>
</para>
<para>
This compressed tarball contains your <filename>debian</filename> directory
contents.  Each and every addition you made to the original source code is
stored as a <command>quilt</command> patch in
<filename>debian/patches</filename>.
</para>
<para>
If someone else wants to re-create your package from scratch, they can easily
do so using the above three files.  The extraction procedure is trivial: just
copy the three files somewhere else and run <literal>dpkg-source -x
gentoo_0.9.12-1.dsc</literal>.  <footnote><para> You can avoid applying
<command>quilt</command> patches in the <literal>3.0 (quilt)</literal> source
format at the end of the extraction with the <literal>--skip-patches</literal>
option.  Alternatively, you can run <literal>dquilt pop -a</literal> after
normal operation.  </para> </footnote>
</para>
</listitem>
<listitem>
<para>
<filename>gentoo_0.9.12-1_i386.deb</filename>
</para>
<para>
This is your completed binary package.  You can use <command>dpkg</command> to
install and remove this just like any other package.
</para>
</listitem>
<listitem>
<para>
<filename>gentoo_0.9.12-1_i386.changes</filename>
</para>
<para>
This file describes all the changes made in the current package revision;
it is used by the Debian FTP archive maintenance programs to install the binary
and source packages.  It is partly generated from the
<filename>changelog</filename> file and the <filename>.dsc</filename> file.
</para>
<para>
As you keep working on the package, its behavior will change and new features will
be added.  People downloading your package can look at this file and quickly
see what has changed.  Debian archive maintenance programs will also post the
contents of this file to the <ulink url="&debian-devel-changes-ldo;">debian-devel-changes@lists.debian.org</ulink>
mailing list.
</para>
</listitem>
</itemizedlist>
<para>
The <filename>gentoo_0.9.12-1.dsc</filename> and
<filename>gentoo_0.9.12-1_i386.changes</filename> files must be signed using
the <command>debsign</command> command with your private GPG key in the
<filename>~/.gnupg/</filename> directory, before uploading them to the Debian
FTP archive. The GPG signature provides the proof that these files are really
yours, using your public GPG key.
</para>
<para>
The <command>debsign</command> command can be made to sign with your specified secret GPG key ID (good for
sponsoring packages) with the following in the <filename>~/.devscripts</filename> file:
</para>
<screen>
DEBSIGN_KEYID=Your_GPG_keyID
</screen>
<para>
The long strings of numbers in the <filename>.dsc</filename> and
<filename>.changes</filename> files are SHA1/SHA256 checksums for the files
mentioned.  Anyone downloading your files can test them with <citerefentry>
<refentrytitle>sha1sum</refentrytitle> <manvolnum>1</manvolnum>
</citerefentry> or <citerefentry> <refentrytitle>sha256sum</refentrytitle>
<manvolnum>1</manvolnum> </citerefentry> and if the numbers don't match,
they'll know the file is corrupt or has been tampered with.
</para>

</section>
<section id="autobuilder"><title>Autobuilder</title>
<para>
Debian supports many <ulink url="&ports;">ports</ulink>
with the <ulink url="&buildd;">autobuilder
network</ulink> running <command>buildd</command> daemons on computers
of many different architectures.  Although you do not need to do this yourself, you
should be aware of what will happen to your packages.  Let's look into roughly
how they rebuild your packages for multiple architectures.
<footnote><para> The actual autobuilder system involves much more complicated
schemes than the one documented here.  Such details are beyond the scope of
this document.  </para> </footnote>
</para>
<para>
For <literal>Architecture: any</literal> packages, the autobuilder system
performs a rebuild.  It ensures the installation of
</para>
<itemizedlist>
<listitem>
<para>
the <systemitem role="package">build-essential</systemitem> package, and
</para>
</listitem>
<listitem>
<para>
packages listed in the <literal>Build-Depends</literal> field (see <xref linkend="control"/>).
</para>
</listitem>
</itemizedlist>
<para>
Then it issues the following command in the source directory:
</para>
<screen>
$ dpkg-buildpackage -B
</screen>
<para>
This will do everything to make architecture dependent binary packages on
another architecture.  It will:
</para>
<itemizedlist>
<listitem>
<para>
clean the source tree (<literal>debian/rules clean</literal>)
</para>
</listitem>
<listitem>
<para>
build the program (<literal>debian/rules build</literal>)
</para>
</listitem>
<listitem>
<para>
build architecture dependent binary packages (<literal>fakeroot debian/rules
binary-arch</literal>)
</para>
</listitem>
<listitem>
<para>
sign the source <filename>.dsc</filename> file, using <command>gpg</command>
</para>
</listitem>
<listitem>
<para>
create and sign the upload <filename>.changes</filename> file, using
<command>dpkg-genchanges</command> and <command>gpg</command>
</para>
</listitem>
</itemizedlist>
<para>
This is why you see your package for other architectures.
</para>
<para>
Although packages listed in the <literal>Build-Depends-Indep</literal> field
are required to be installed for our normal packaging work (see
<xref linkend="completebuild"/>), they are not required to be installed for the
autobuilder system since it builds only architecture dependent binary packages.
<footnote><para> Unlike under the <systemitem role="package">pbuilder</systemitem> package, the <command>chroot</command>
environment under the <systemitem role="package">sbuild</systemitem> package
used by the autobuilder system does not enforce the use of a minimal
system and may have many leftover packages installed.  </para>
</footnote> This distinction between normal packaging and autobuilding
procedures is what dictates whether you should record such required
packages in the <literal>Build-Depends</literal> or
<literal>Build-Depends-Indep</literal> fields of the
<filename>debian/control</filename> file (see <xref linkend="control"/>).
</para>
</section>
<section id="debuild"><title><command>debuild</command> command</title>
<para>
You can automate the build activity around executing the <command>dpkg-buildpackage</command> command
package further with the <command>debuild</command> command.  See <citerefentry>
<refentrytitle>debuild</refentrytitle> <manvolnum>1</manvolnum>
</citerefentry>.
</para>
<para>
The <command>debuild</command> command executes the <command>lintian</command> command to make a static check after building the Debian package.
The <command>lintian</command> command can be customized with the following in the <filename>~/.devscripts</filename> file:
</para>
<screen>
DEBUILD_DPKG_BUILDPACKAGE_OPTS="-us -uc -I -i"
DEBUILD_LINTIAN_OPTS="-i -I --show-overrides"
</screen>
<para>
Cleaning the source and rebuilding the package from your user account is as simple as:
</para>
<screen>
$ debuild
</screen>
<para>
You can clean the source tree as simply as:
</para>
<screen>
$ debuild -- clean
</screen>
</section>
<section id="pbuilder"><title><systemitem role="package">pbuilder</systemitem> package</title>
<para>
For a clean room (<command>chroot</command>) build environment to verify the
build dependencies, the <systemitem role="package">pbuilder</systemitem>
package is very useful.  <footnote><para> Since the <systemitem
role="package">pbuilder</systemitem> package is still evolving, you should
check the actual configuration situation by consulting the latest official
documentation.</para> </footnote> This ensures a clean build from the source
under the <literal>sid</literal> auto-builder for different architectures and
avoids a severity serious FTBFS (Fails To Build From Source) bug which is
always in the RC (release critical) category. 
<footnote><para>See <ulink url="&buildd-do;"/> for more on
Debian package auto-building.</para></footnote>
</para>
<para>
Let's customize the <systemitem role="package">pbuilder</systemitem> package as
follows:
</para>
<itemizedlist>
<listitem>
<para>
setting the <filename>/var/cache/pbuilder/result</filename> directory writable by your user account.
</para>
</listitem>
<listitem>
<para>
creating a directory, e.g.
<filename><replaceable>/var/cache/pbuilder/hooks</replaceable></filename>,
writable by the user, to place hook scripts in.
</para>
</listitem>
<listitem>
<para>
configuring <filename>~/.pbuilderrc</filename> or
<filename>/etc/pbuilderrc</filename> to include the following.
</para>
<screen>
AUTO_DEBSIGN=${AUTO_DEBSIGN:-no}
HOOKDIR=<replaceable>/var/cache/pbuilder/hooks</replaceable>
</screen>
</listitem>
</itemizedlist>
<para>
First let's initialize the local <systemitem role="package">pbuilder</systemitem> <command>chroot</command> system as
follows:
</para>
<screen>
$ sudo pbuilder create
</screen>
<para>
If you already have a completed source package, issue the following commands
in the directory where the
<filename><replaceable>foo</replaceable>.orig.tar.gz</filename>,
<filename><replaceable>foo</replaceable>.debian.tar.gz</filename>, and
<filename><replaceable>foo</replaceable>.dsc</filename> files exist to update
the local <systemitem role="package">pbuilder</systemitem>
<command>chroot</command> system and to build binary packages in it:
</para>
<screen>
$ sudo pbuilder --update
$ sudo pbuilder --build <replaceable>foo_version</replaceable>.dsc
</screen>
<para>
The newly built packages without the GPG signatures will be located in
<filename>/var/cache/pbuilder/result/</filename> with non-root ownership.
</para>
<para>
The GPG signatures on the <filename>.dsc</filename> file and the
<filename>.changes</filename> file can be generated as:
</para>
<screen>
$ cd /var/cache/pbuilder/result/
$ debsign <replaceable>foo_version_arch</replaceable>.changes
</screen>
<para>
If you have an updated source tree but have not generated the matching
source package, issue the following commands in the source directory where the
<filename>debian</filename> directory exists, instead:
</para>
<screen>
$ sudo pbuilder --update
$ pdebuild
</screen>
<para>
You can log into its <command>chroot</command> environment with the
<literal>pbuilder --login --save-after-login</literal> command and configure it
as you wish.  This environment can be saved by leaving its shell prompt with
<literal>^D</literal> (Control-D).
</para>
<para>
The latest version of the <command>lintian</command> command can be executed in
the <literal>chroot</literal> environment using the hook script
<filename><replaceable>/var/cache/pbuilder/hooks</replaceable>/B90lintian</filename>
configured as follows:  <footnote><para> This assumes
<literal>HOOKDIR=/var/cache/pbuilder/hooks</literal>.  You can find many
examples of hook scripts in the
<filename>/usr/share/doc/pbuilder/examples</filename> directory.  </para>
</footnote>
</para>
<screen>
#!/bin/sh
set -e
install_packages() {
        apt-get -y --allow-downgrades install "$@"
        }
install_packages lintian
echo "+++ lintian output +++"
su -c "lintian -i -I --show-overrides /tmp/buildd/*.changes" - pbuilder
# use this version if you don't want lintian to fail the build
#su -c "lintian -i -I --show-overrides /tmp/buildd/*.changes; :" - pbuilder
echo "+++ end of lintian output +++"
</screen>
<para>
You need to have access to the latest <literal>sid</literal> environment to
build packages properly for <literal>sid</literal>.  In practice,
<literal>sid</literal> may be experiencing issues which makes it undesirable
for you to migrate your whole system.  The <systemitem role="package">pbuilder</systemitem> package can help you to cope with this
kind of situation.
</para>
<para>
You may need to update your <literal>stable</literal> packages after their
release for <literal>stable-proposed-updates</literal>,
<literal>stable/updates</literal>, etc.  <footnote><para> There are some
restrictions for such updates of your <literal>stable</literal> package.
</para> </footnote> For such occasions, the fact that you may be running a <literal>sid</literal>
system is not a good enough excuse for failing to update them promptly.  The <systemitem role="package">pbuilder</systemitem> package can help you to access
environments of almost any Debian derivative distribution of the same 
architecture.
</para>
<para>
See <ulink url="&pbuilder;"/>,
<citerefentry> <refentrytitle>pdebuild</refentrytitle> <manvolnum>1</manvolnum>
</citerefentry>, <citerefentry> <refentrytitle>pbuilderrc</refentrytitle>
<manvolnum>5</manvolnum> </citerefentry>, and <citerefentry>
<refentrytitle>pbuilder</refentrytitle> <manvolnum>8</manvolnum>
</citerefentry>.
</para>
</section>
<section id="git-buildpackage"><title><command>git-buildpackage</command> command and similar</title>
<para>
If your upstream uses a source code management system (VCS)
<footnote><para>See <ulink url="&debref-vcs;">Version control systems</ulink> for more.</para></footnote>
to maintain their code, you should consider using it as well.  This makes merging
and cherry-picking upstream patches much easier.  There are several specialized
wrapper script packages for Debian package building for each VCS.
</para>
<itemizedlist>
<listitem>
<para>
<systemitem role="package">git-buildpackage</systemitem>: a suite to help with
Debian packages in Git repositories.
</para>
</listitem>
<listitem>
<para>
<systemitem role="package">svn-buildpackage</systemitem>: helper programs to
maintain Debian packages with Subversion.
</para>
</listitem>
<listitem>
<para>
<systemitem role="package">cvs-buildpackage</systemitem>: a set of Debian
package scripts for CVS source trees.
</para>
</listitem>
</itemizedlist>
<para>
Use of <systemitem role="package">git-buildpackage</systemitem> is becoming quite popular for Debian Developers to manage Debian packages with the Git server on <ulink url="&debian-alioth;">alioth.debian.org</ulink>. <footnote><para><ulink url="&debian-alioth-wiki;">Debian wiki Alioth</ulink> documents how to use the <ulink url="&debian-alioth;">alioth.debian.org</ulink>  service.</para></footnote> This package offers many commands to <emphasis>automate</emphasis> packaging activities:
</para>
<itemizedlist>
<listitem> <para> <citerefentry> <refentrytitle>gbp-import-dsc</refentrytitle> <manvolnum>1</manvolnum> </citerefentry>: import a previous Debian package to a Git repository.</para> </listitem>
<listitem> <para> <citerefentry> <refentrytitle>gbp-import-orig</refentrytitle> <manvolnum>1</manvolnum> </citerefentry>: import a new upstream tar to a Git repository.</para> </listitem>
<listitem> <para> <citerefentry> <refentrytitle>gbp-dch</refentrytitle> <manvolnum>1</manvolnum> </citerefentry>: generate the Debian changelog from Git commit messages.</para> </listitem>
<listitem> <para> <citerefentry> <refentrytitle>git-buildpackage</refentrytitle> <manvolnum>1</manvolnum> </citerefentry>: build Debian packages from a Git repository.</para> </listitem>
<listitem> <para> <citerefentry> <refentrytitle>git-pbuilder</refentrytitle> <manvolnum>1</manvolnum> </citerefentry>: build Debian packages from a Git repository using <command>pbuilder</command>/<command>cowbuilder</command>.</para> </listitem>
</itemizedlist>
<para>
These commands use 3 branches to track packaging activity:
</para>
<itemizedlist>
<listitem> <para> <literal>main</literal> for Debian package source tree.</para> </listitem>
<listitem> <para> <literal>upstream</literal> for upstream source tree.</para> </listitem>
<listitem> <para> <literal>pristine-tar</literal> for upstream tarball generated by the <literal>--pristine-tar</literal> option.<footnote><para>The <literal>--pristine-tar</literal> option invokes the <command>pristine-tar</command> command, which can regenerate an exact copy of a pristine upstream tarball using only a small binary delta file and the contents of the tarball that are typically kept in an <literal>upstream</literal> branch in the VCS. </para></footnote> </para> </listitem>
</itemizedlist>
<para>
You can configure <systemitem role="package">git-buildpackage</systemitem> with <filename>~/.gbp.conf</filename>. See <citerefentry> <refentrytitle>gbp.conf</refentrytitle> <manvolnum>5</manvolnum> </citerefentry>.
<footnote><para> Here are some web resources available for advanced audiences.  </para>
<itemizedlist>
<listitem> <para> Building Debian Packages with git-buildpackage (<filename>&git-buildpackage-doc;</filename>)</para> </listitem>
<listitem> <para> <ulink url="&debian-packages-git;">debian packages in git</ulink> </para> </listitem>
<listitem> <para> <ulink url="&git-debian-packaging;">Using Git for Debian Packaging</ulink> </para> </listitem> 
<listitem> <para> <ulink url="&git-dpm;">git-dpm: Debian packages in Git manager</ulink> </para> </listitem>
</itemizedlist> 
</footnote>
</para>
</section>

<section id="quickrebuild"><title>Quick rebuild</title>
<para>
With a large package, you may not want to rebuild from scratch every time while
you're tuning details in <filename>debian/rules</filename>.  For testing purposes,
you can make a <filename>.deb</filename> file without rebuilding the upstream
sources like this<footnote><para> Environment variables that are normally
configured to proper values are not set by this method.  Never create real
packages to be uploaded using this <emphasis role="strong">quick</emphasis>
method.  </para> </footnote>:
</para>
<screen>
$ fakeroot debian/rules binary
</screen>
<para>
Or simply do the following to see if it builds or not:
</para>
<screen>
$ fakeroot debian/rules build
</screen>
<para>
Once you are finished with your tuning, remember to rebuild following the
proper procedure.  You may not be able to upload correctly if you try to upload
<filename>.deb</filename> files built this way.
</para>
</section>
<section id="hierarchy"><title>Command hierarchy</title>
<para>
Here is a quick summary of how many commands to build packages fit together in the command hierarchy. There are many ways to do the same thing.
</para>
<itemizedlist>
<listitem><para><filename>debian/rules</filename> = maintainer script for the package building</para> </listitem>
<listitem><para><command>dpkg-buildpackage</command> = core of the package building tool</para> </listitem>
<listitem><para><command>debuild</command> = <command>dpkg-buildpackage</command> + <command>lintian</command> (build under the sanitized environment variables)</para> </listitem>
<listitem><para><command>pbuilder</command> = core of the Debian chroot environment tool</para> </listitem>
<listitem><para><command>pdebuild</command> = <command>pbuilder</command> + <command>dpkg-buildpackage</command> (build in the chroot)</para> </listitem>
<listitem><para><command>cowbuilder</command> = speed up the <command>pbuilder</command> execution</para> </listitem>
<listitem><para><command>git-pbuilder</command> = the easy-to-use commandline syntax for <command>pdebuild</command> (used by <command>gbp buildpackage</command>)</para> </listitem>
<listitem><para><command>gbp</command> = manage the Debian source under the git repo</para> </listitem>
<listitem><para><command>gbp buildpackage</command> = <command>pbuilder</command> + <command>dpkg-buildpackage</command> + <command>gbp</command></para> </listitem>
</itemizedlist>
<para>
Although use of higher level commands such as <command>gbp buildpackage</command> and <command>pbuilder</command> ensures the perfect package building environment, it is essential to understand how lower level commands such as <filename>debian/rules</filename>  and <command>dpkg-buildpackage</command> are executed under them.
</para>
</section>
</chapter>