1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
|
# The GAP build system
This file is meant to give an overview of how the GAP build system works. It
is targeted at people who need to work on the build system itself (to extend
it, fixes bugs in it, etc.). It should not be necessary to read this if all
you want to do is compile GAP and work on the GAP library and kernel.
Note that this really is just an overview; for details, please refer to the
comments inside the various parts of the build system.
## Prerequisites
In order to work on the build system, you need at least the following:
* GNU autoconf (we recommend 2.69 or later)
* GNU make
Note that we extensively use features provided by GNU make, so in general
another version of make, such as BSD make, is not suitable.
## Quick start: building GAP with no frills
If you are working with a fresh clone of the GAP repository, you need to
run the `autogen.sh` script first, which will generate the `configure`
script. Afterwards, or if you are using a release version of GAP, you
can follow the standard procedure:
./configure
make
== Overview of the files constituting the GAP build system
* `autogen.sh`: sets up the build system; typically the first thing to run in
a fresh clone of the GAP repository. It runs `autoconf` and `autoheader`.
* `configure`: generated by `autogen.sh` from `configure.ac`.
* `configure.ac`: the GNU autoconf source of our configure script.
* `GNUmakefile`, `GNUmakefile.in`: The former file is generated from the
latter by `configure`. It is the primary Makefile (GNU make prefers it
over `Makefile`). It only contains variables and vpath settings, and
includes `Makefile.rules` for the actual build rules.
* `Makefile`: This is a placeholder file, and serves two purposes:
1. If the user runs `make` before `configure`, it prints a warning.
2. If `configure` did run, but `make` is not GNU make, it produces
a corresponding error message.
* `Makefile.rules`: This is the core of the build system. If you want
to add or remove a kernel C source file, you need to add or remove
its name here and only here.
* `cnf/`: All files in this directory are part of the build system.
* `extern/`: External libraries we bundle with GAP (such as GMP) are
put in here.
* `build/`: Generated code (such as `config.h` and `version.c`) is put
into this directory.
* `build/obj/`: All `*.o` resp. `*.lo` files are placed into this directory.
- `build/deps/` directories contain `*.d` files generated by the build system,
and which are used to track dependencies, e.g. of C source files on header
files.
## Out-of-tree builds
The old GAP build system had a concept of "configs" and "CONFIGNAME", which
allowed you to build GAP in different configurations from a single set of
sources. This is gone in the current build system. However, a similar goal can
be achieved by using so-called "out-of-tree builds".
In the following and also in the files that make up the build system, "srcdir"
refers to the directory containing the GAP files, i.e. it contains this
README, the src and lib directories and more.
To create a new out-of-tree build, create a new directory anywhere
in your filesystem. A typical setup places the out-of-tree dirs into
subdirectories of a "build" directory inside the srcdir. So you might
have directories
srcdir/build/default
srcdir/build/default32
srcdir/build/hpcgap
srcdir/build/hpcgap32
...
We will refer to this directory from now on as the "builddir".
To initialize the out-of-tree build, change into the builddir and
execute the configure script from the srcdir, like this:
cd $builddir
$srcdir/configure
You can pass any additional options you like to configure, e.g. `ABI=32`
or `--enable-hpcgap`.
Once the configure script has completed, you can run `make` as usual,
and all the object files and the gap executable will be placed inside
builddir. Your srcdir will remain untouched.
## Dependency tracking
The build system tracks dependencies between files, such as between C source and
header files, via `*.d` files stored in `build/deps/`.
These files are mostly generated by the compiler; for this, the compiler needs
to support the relevant flags (gcc, clang, icc all do so).
For a detailed explanation of a very similar scheme, see here:
<https://make.mad-scientist.net/papers/advanced-auto-dependency-generation/>
## HPC-GAP integration
One of the main features of the new build system is that it optionally allows
to build HPC-GAP instead of plain GAP. HPC-GAP is an experimental fork of GAP
which implements concurrent programming, multi-threading, etc..
The HPC-GAP kernel and library were forked from the GAP kernel and library and
developed semi-independently for several years, with occasional merges between
the two. In order to recombine the two, we merged the HPC-GAP fork into a
subdirectory `hpcgap` of the GAP repository. Then, all files inside `hpcgap`
which were identical to their counterparts in the GAP repository were deleted
(e.g. `hpcgap/src/ariths.c` was deleted as it was identical to `src/ariths.c`).
At this point, `hpcgap/src` has been fully merged, but there are still files
in `hpcgap/lib/` which differ from their counterparts in `lib/`
The new build system can optionally be instructed to build HPC-GAP, by
passing the `--enable-hpcgap` flag to the `configure` script. For the
resulting HPC-GAP binary to work, a trick is used: HPC-GAP mode uses multiple
GAP root paths. Specifically, the GAP kernel function `SySetGapRootPath` was
modified so that for every root directory `FOO` that gets added, we first add
`FOO/hpcgap` to the list of root directories. This way, `GAPROOT/hpcgap/lib`
is searched first for files, and only if no matching file is found there does
GAP also search in `GAPROOT/lib`.
## Cross compilation
The GAP build system supports cross compilation by leveraging the GNU autoconf
support for this (please consult the GNU autoconf manual for details).
When building GAP from a release archive, everything should just work (if
it doesn't, please report it to us as a bug).
However, for development versions of GAP built directly from its git sources,
there is a complication: GAP uses a few C source files which are the output of
the GAP-to-C compiler `gac`, which in turn needs a working `gap` executable.
This requires a bootstrapping process, which works because GAP can actually be
built and used without those files, they are merely a performance
optimization. So what our build system does is to first compile a version of
GAP without those files, then run that to generate those files (if they are
missing or not up-to-date). Then finally the actual GAP executable is compiled.
Unfortunately, this poses a problem for cross compilation: when compiling GAP
on one computer (the "build" architecture) for a different computer with a
different OS / CPU architecture / whatnot (the "host" architecture), then the
first GAP executable needs to be built for the "build" architecture, so that
it can run during the build process and generated code; while the second GAP
executable needs to be compiled for the "host" architecture.
A second cross compilation obstacle is that there files `build/ffdata.c` and
`build/ffdata.h` are generated by a C program `ffgen` that thus needs to be
built for the "build" architecture.
To overcome these obstacles, you may proceed as follows:
1. First build GAP for the "build" architecture as usual.
2. This produces files `build/c_*.c` and `build/ffdata.*` which you should
copy into the `src` directory.
3. Clean all build artifacts (alternatively, perform steps 1 & 2 in an
out-of-tree build directory)
3. Build GAP a second time, for the "host" architecture. The build system
will detect the files you copied into `src` in step 2, and will then not
attempt to generate them.
Here is a minimal example to illustrate this:
./configure && make # build native GAP
cp build/c_*.c build/ffdata.* src # copy the generated code
make clean
./configure --host=HOST_ARCH # build for the "host" architecture
make
Note that this is really a minimalistic example; for actual cross compilation,
you may need to provide further flags to the `configure` script and/or set
environment variables, e.g. to ensure it finds dependencies like GMP or GNU
readline, or to specify an installation prefix, and so on.
One final remark: some of the generated files differ for HPC-GAP. If you want
to cross compile HPC-GAP, you need to generate them with HPC-GAP, and place
them into `src/hpc` instead of `src`, like in this example:
./configure --enable-hpcgap
make
cp build/c_*.c src/hpc/
cp build/ffdata.* src/
make clean
./configure --host=HOST_ARCH --enable-hpcgap
make
## Open tasks
There are many things that still need to be done in the new build system. For
an overview, see
<https://github.com/gap-system/gap/issues?q=is%3Aopen+is%3Aissue+label%3A%22topic%3A+build+system%22>
|
