1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
|
Abstract
########
**mini-buildd** is a custom build daemon for *Debian*-based distributions with *all batteries included*: I.e., it covers
**incoming**, **(distributed) building**, **installing**, **repository maintenance**, **repository delivery** -- and
some extras.
Main notion is to make it easy to bootstrap and maintain an add-on (i.e., add onto an existing Debian distribution)
Debian package infrastructure. So it's, for example, suitable for *sandboxing*, *personal package repositories* or
*organizations* maintaining custom or adapted Debian packages.
mini-buildd does not re-invent the wheel, it's rather a sophisticated glue written in ``python3`` to a number of
standard (Debian) or off-the-shelf software components. As core components, it uses :debpkg:`twisted` (HTTP server),
:debpkg:`python-django` (web application), :debpkg:`python-pyftpdlib` (package uploads), :debpkg:`sbuild` &&
:debpkg:`schroot` (package building) and :debpkg:`reprepro` (for the APT package archive).
Features
********
*Core features*:
* Integrated **HTTP server** (webapp and repository delivery).
* Integrated **FTP server** (incoming).
* Easy bootstrap or maintenance via :apicall:`setup`, fine-grained custom setup via **Web-based configuration**.
* **Distributed builders** (with optional wake support for remotes).
* **Web-based** or **command line** repository maintenance via *API* calls.
*Some Arbitrary Extras*:
* *Events Queue*: Web or shell, check what's going on.
* *User Management*: Package subscriptions, GPG key management, upload authorization.
* *Package QA*: Internal sanity checks, version enforcing, lintian, autopkgtest -- we make package building hard again.
* *Package Tracking*: ``Debian PTS``-(sort-of)-like web based source package tracker.
* *No-Changes-Ports*: Automates ports, for internal or external source packages.
* *Keyring Packages*: Generated automatically.
* *Package Staging*: ``unstable->testing->stable``-style (manual) migration.
* *Package Rollbacks*: Keeps ``N`` rollbacks for any distribution.
* *Repository Snapshots*: Automated (time based) or manual.
Basic Mode Of Operation
***********************
**mini-buildd** is a Unix daemon written in python. When running, it provides a HTTP server (on port ``8066`` by
default).
The HTTP server serves both, mini-buildd's web application as well as the delivery of the package repositories.
The instance is being configured in the configuration section of the web application.
As soon as a *mini-buildd* instance has been configured to have an active 'Daemon', you may ``start`` the engine,
running an FTP server (on port ``8067`` by default).
The FTP server acts on incoming ``*.changes`` files, both from developers and other mini-buildd instances (via special
``buildrequest`` and ``buildresult`` changes).
As soon as an instance of *mini-buildd* has active chroots configured, it acts as *builder*. Chroots are completely
generic and interchangeable, and identified by *codename* and *arch* only; distribution-specific build configuration is
all carried through the internal buildrequests. Thus, *mini-buildd* instances may be interconnected as so-called
'Remotes' to share builders.
This is a simplified example mini-buildd 'network' with three mini-buildd instances *ernie*, *grover* and *bert*:
.. graphviz::
digraph flow_simple
{
node [fontname=Arial fontsize=11 shape=diamond style=filled fillcolor=grey];
edge [fontname=Helvetica fontsize=8];
subgraph cluster_0
{
style=filled;
color=lightgrey;
label="ernie";
"Ernie-Packager" [label="Packager"];
"Ernie-Builder" [label="Builder"];
"Ernie-Repositories" [label="Repositories" shape=folder];
}
"Ernie-Developer" [shape=oval fillcolor=lightgrey];
"Ernie-Developer" -> "Ernie-Packager" [label="uploads"];
"Ernie-Packager" -> "Ernie-Repositories" [label="installs"];
"Ernie-Packager" -> {"Ernie-Builder" "Grover-Builder"} [dir=both label="builds"];
"Ernie-Manager" [shape=oval fillcolor=lightgrey];
"Ernie-Manager" -> "Ernie-Repositories" [label="manages"];
"Ernie-User" [shape=oval fillcolor=lightgrey];
"Ernie-Repositories" -> "Ernie-User" [label="apt"];
subgraph cluster_1
{
style=filled;
color=lightgrey;
label="grover";
"Grover-Builder" [label="Builder"];
}
subgraph cluster_2
{
style=filled;
color=lightgrey;
label="bert";
"Bert-Packager" [label="Packager"];
"Bert-Repositories" [label="Repositories" shape=folder];
}
"Bert-Developer" [shape=oval fillcolor=lightgrey];
"Bert-Developer" -> "Bert-Packager" [label="uploads"];
"Bert-Packager" -> "Bert-Repositories" [label="installs"];
"Bert-Packager" -> {"Ernie-Builder" "Grover-Builder"} [dir=both label="builds"];
"Bert-Manager" [shape=oval fillcolor=lightgrey];
"Bert-Manager" -> "Bert-Repositories" [label="manages"];
"Bert-User" [shape=oval fillcolor=lightgrey];
"Bert-Repositories" -> "Bert-User" [label="apt"];
}
* *ernie* has repositories and chroots, and uses himself and *grover* as remote for building.
* *grover* only has chroots, and is used by *ernie* and *bert* for building.
* *bert* only has repositories, and uses *ernie* and *grover* as remotes for building.
About this Manual
*****************
This manual is neither complete nor the only source of documentation, especially for the nitty-gritty details:
* Command usages (a.k.a. ``--help``) (or resp. manpages -- same content). Copies are in this manual, see sidebar.
* API call documentation (see :mbdpage:`api`). Manual provides resp. links to In-App-Docs, see sidebar.
* Python code doc (see :mod:`mini_buildd`).
* In-code django docs for the models (see :mbdpage:`setup`, :mbdpage:`admin`).
* And of course as final endboss, you still have the ``source code``.
.. tip:: Some links do not work in this manual?
If you are reading this manual stand-alone the resp. links to "In-App content" will not work.
Read this manual served by mini-buildd itself.
Conventions
===========
Sections
--------
Section hierarchy ``#*=-~``:
* ``###``: Document
* ``***``: Section
* ``===``: Subsection
* ``---``: Subsubsection
* ``~~~``: Subsubsubsection
Admonitions
-----------
We use RST admonitons ``error``, ``attention``, ``note``, ``tip`` (so these can be summarized in :doc:`auto_admonitions`).
Code
----
In code snippets, names written all-capital (``FOO``, ``HOST``) are not meant literal but placeholders for customized
values.
Code expected to run as ``any user``:
.. code:: bash
mini-buildd-dput foo bar.dsc
Code expected to be run as ``mini-buildd``:
.. code:: bash
:class: mini-buildd-code
mini-buildd-cruft
Code expected to be run as ``root``:
.. code:: bash
:class: root-code
apt install foo
|
