1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283
|
#!/usr/bin/make -f
BUILDER_IMAGES= \
almalinux-8 \
almalinux-9 \
centos-stream9 \
fedora-latest \
fedora-rawhide \
debian-stretch \
debian-bullseye \
debian-bookworm \
debian-testing \
debian-sid \
ubuntu-focal \
ubuntu-jammy \
ubuntu-noble \
ubuntu-lunar \
ubuntu-mantic \
kira \
tarball
IMAGES=${BUILDER_IMAGES} devshell
DEFAULT_IMAGE ?= ubuntu-noble
TARBALL_IMAGE ?= tarball
DEFAULT_DEB_IMAGE=ubuntu-noble
DEFAULT_RPM_IMAGE=almalinux-8
DOCKER=docker
CONTAINER_REGISTRY ?= ghcr.io/syslog-ng
MODE ?= snapshot
VERSION ?= $(shell MODE=${MODE} scripts/version.sh)
DOCKER_RUN_ARGS=-e USER_NAME_ON_HOST=$(shell whoami) \
--network=host --privileged \
--ulimit nofile=1024:1024 \
-v $(ROOT_DIR):/source \
-v $(DBLD_DIR):/dbld \
-v $(DBLD_DIR)/build:/build \
-v $(DBLD_DIR)/install:/install \
-e CONFIGURE_OPTS="$${CONFIGURE_OPTS:-$(CONFIGURE_OPTS)}" \
-e CCACHE_DIR=/build/ccache \
-e MODE=$(MODE) \
-e VERSION=$(VERSION) \
-e PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin$(if $(DOCKER_EXTRA_PATH),:$(DOCKER_EXTRA_PATH)) \
-e GRADLE_USER_HOME=/build/gradle-home \
-e GRADLE_PROJECT_CACHE_DIR=/build/gradle-cache \
-e GRADLE_FLAGS=--build-cache \
$(if $(wildcard ${HOME}/.gitconfig),-v ${HOME}/.gitconfig:/build/.gitconfig)
DOCKER_BUILD_ARGS=
ROOT_DIR=$(shell pwd)
DBLD_DIR=$(ROOT_DIR)/dbld
BUILD_DIR=$(DBLD_DIR)/build
RELEASE_DIR=$(DBLD_DIR)/release
TARBALL_BASENAME=syslog-ng-$(VERSION).tar.gz
TARBALL=$(BUILD_DIR)/$(TARBALL_BASENAME)
GIT=$(shell which git || echo false)
GIT_RELEASE_TAG=syslog-ng-$(VERSION)
CONFIGURE_OPTS=--enable-debug --enable-manpages --with-python=3 --prefix=/install $(CONFIGURE_ADD)
DBLD_RULES=$(MAKE) --no-print-directory -f $(DBLD_DIR)/rules
DOCKER_INTERACTIVE=$(shell if tty -s; then echo "-ti"; else echo "-i"; fi)
DOCKER_SHELL=$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -${DOCKER_INTERACTIVE} ${CONTAINER_REGISTRY}/dbld-$* /dbld/shell $(if $(SHELL_COMMAND),"$(SHELL_COMMAND)",bash)
-include $(if $(RULES_CONF),$(RULES_CONF),$(DBLD_DIR)/rules.conf)
help:
@echo "This script allows you to build release/snapshot artifacts, such "
@echo "as rpm, deb packages and tarballs in controlled environments (e.g. "
@echo "within docker containers)."
@echo ""
@echo "Please create the required docker image first using the 'image' target"
@echo "then you can build syslog-ng binaries or do development using the "
@echo "targets described below."
@echo ""
@echo "Supported OSs are: $(IMAGES)"
@echo ""
@echo "Packaging targets:"
@echo " deb: generate debs in dbld/build for $(DEFAULT_DEB_IMAGE)"
@echo " rpm: generate RPMs in dbld/build for $(DEFAULT_RPM_IMAGE)"
@echo " deb-<os>: generate debs in dbld/build for the specified OS"
@echo " rpm-<os>: generate rpms in dbld/build for the specified OS"
@echo ""
@echo "Docker images:"
@echo " image: generate the default docker image"
@echo " image-<os>: generate docker image with a specific OS for building syslog-ng"
@echo " images: generate all docker images"
@echo ""
@echo "Development:"
@echo " bootstrap: bootstrap source from a git clone (e.g. autogen.sh and friends)"
@echo " shell: start a shell in the specified OS"
@echo " login: start a 2nd shell alongside an already running one"
@echo " root-login: start a 2nd shell (as root) alongside an already running one"
@echo " tarball: generate a tarball in a controlled environment (e.g. docker)"
@echo " pkg-tarball: generate a tarball that includes deb/rpm packaging files"
@echo " prepare-release VERSION=x.y.z: prepare the source tree for release, does not commit"
@echo " release VERSION=x.y.z: generate a release tarball/deb package in a controlled environment (e.g. docker)"
@echo ""
@echo "Notable Make variables:"
@echo " CONFIGURE_ADD -- add this to the configure command line in bootstrap"
@echo " DEFAULT_IMAGE -- override the default image, currently $(DEFAULT_IMAGE)"
@echo " TARBALL_IMAGE -- override the default image where the tarball is generated, currently $(TARBALL_IMAGE)"
@echo ""
bootstrap: bootstrap-$(DEFAULT_IMAGE)
bootstrap-%: setup
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/bootstrap
make-%: setup
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$(DEFAULT_IMAGE) /dbld/make $(MAKE_ARGS) $*
tarball-from-root: setup
@if [ -f $(ROOT_DIR)/../$(TARBALL_BASENAME) ] && [ ! -f $(TARBALL) ]; then \
cp $(ROOT_DIR)/../$(TARBALL_BASENAME) $(TARBALL); \
fi
tarball: tarball-$(TARBALL_IMAGE)
tarball-%: tarball-from-root
@echo "Checking $(TARBALL) if it is up-to-date..."; \
if [ -f $(TARBALL) ]; then \
TARBALL_CHANGES=`find $(ROOT_DIR) -newer $(TARBALL) | sed -e 's,^$(ROOT_DIR),,' | grep -v -f $(DBLD_DIR)/tarball-changes.ignore | tee $(BUILD_DIR)/tarball-changed-files.txt | wc -l`; \
fi; \
if [ ! -f $(TARBALL) ] || [ "$${TARBALL_CHANGES}" -gt 0 ]; then \
if [ -f $(TARBALL) ]; then \
echo "Rebuilding, because these contents have changed since the tarball" && \
cat $(BUILD_DIR)/tarball-changed-files.txt; \
else \
echo "Rebuilding, because tarball $(TARBALL) not found"; \
fi; \
echo "Git status follows:" && \
( $(GIT) status || echo "Git not found..." ) && \
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/tarball; \
else \
echo "Tarball $(TARBALL) is up to date (except files excluded by $(DBLD_DIR)/tarball-changes.ignore)"; \
fi
pkg-tarball: pkg-tarball-$(TARBALL_IMAGE)
#
# pkg-tarball-%:
#
# NOTE: first check if the tarball is already a pkg-tarball (containing
# packaging files). This might happen if we try to reuse a cached tarball
# and we can spare some time by avoiding the pulling of the tarball image as
# well as not having to run the scripts that add the packaging files
#
pkg-tarball-%: tarball-%
@if ! tar --strip-components=1 --show-transformed-names -tvf $(TARBALL) | grep ' debian/rules$$' > /dev/null; then \
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/pkg-tarball; \
fi
package: package-$(DEFAULT_IMAGE)
package-%: pkg-tarball
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/package
deb: deb-$(DEFAULT_DEB_IMAGE)
deb-%: pkg-tarball
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/deb
rpm: rpm-$(DEFAULT_RPM_IMAGE)
rpm-%: pkg-tarball
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$* /dbld/rpm
validate-tree-clean:
@if ! $(GIT) diff-index --quiet HEAD; then \
echo "Your source tree has changed, this operation requires a clean git tree."; \
exit 1; \
fi
validate-version-format:
@if [ "$$(echo $(VERSION) | sed -e 's/^[0-9]\+\.[0-9]\+.[0-9]\+//')" != "" ]; then \
echo "The version number you specified $(VERSION) is not a valid version for a RELEASE. Please supply one using VERSION= via the command line"; \
exit 1; \
fi
prepare-release: setup validate-tree-clean validate-version-format
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$(TARBALL_IMAGE) /dbld/prepare-release $(VERSION)
validate-release: validate-tree-clean validate-version-format
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm -i ${CONTAINER_REGISTRY}/dbld-$(TARBALL_IMAGE) /dbld/validate-release-version $(VERSION)
@if $(GIT) rev-parse --verify -q "$(GIT_RELEASE_TAG)" > /dev/null; then \
echo "Your git tree already has $(GIT_RELEASE_TAG), this might indicate a duplicate release, please remove that first."; \
exit 1; \
fi
#
# release:
#
# the order of these is important, so invoke ourselves recursively
# instead of using make dependencies, which can be reordered in parallel builds
#
release: validate-release
ARTIFACT_DIR=$(RELEASE_DIR)/$(VERSION) && \
rm -rf "$$ARTIFACT_DIR" && mkdir -p "$$ARTIFACT_DIR" && rm -rf $(TARBALL) && \
$(DBLD_RULES) MODE=release VERSION=$(VERSION) pkg-tarball && \
echo "Building the release was successful, artifacts stored in $$ARTIFACT_DIR" && \
$(DBLD_RULES) MODE=release VERSION=$(VERSION) tag-release
tag-release:
$(GIT) tag $(GIT_RELEASE_TAG)
clean:
rm -rf $(BUILD_DIR)/*
run: run-$(DEFAULT_IMAGE)
run: RUN_COMMAND=echo Specify RUN_COMMAND to do something sensible here
run-%: setup
$(DOCKER) run $(DOCKER_RUN_ARGS) --rm ${DOCKER_INTERACTIVE} ${CONTAINER_REGISTRY}/dbld-$* bash -c "$(RUN_COMMAND)"
shell: shell-$(DEFAULT_IMAGE)
shell-%: setup
$(DOCKER_SHELL)
images: $(foreach image,$(IMAGES), image-$(image))
image: image-$(DEFAULT_IMAGE)
image-kira: image-ubuntu-focal
image-devshell: image-debian-bookworm
image-tarball: image-debian-bookworm
image-%:
$(DBLD_DIR)/prepare-image-build $* && \
$(DOCKER) build $(DOCKER_BUILD_ARGS) --build-arg=ARG_IMAGE_PLATFORM=$* --build-arg=COMMIT=$$($(GIT) rev-parse --short HEAD || echo "") --build-arg=CONTAINER_REGISTRY=${CONTAINER_REGISTRY} --network=host -t ${CONTAINER_REGISTRY}/dbld-$* -f $(DBLD_DIR)/images/$*.dockerfile $(DBLD_DIR)
push-images: $(foreach image,$(IMAGES), push-image-$(image))
push-image-%:
@echo "Pushing image: $*"
$(DOCKER) push ${CONTAINER_REGISTRY}/dbld-$*
pull-images: $(foreach image,$(BUILDER_IMAGES), pull-image-$(image))
pull-image: pull-image-$(DEFAULT_IMAGE)
pull-image-%:
$(DOCKER) pull ${CONTAINER_REGISTRY}/dbld-$*
cache-image-%:
@IMAGE=${CONTAINER_REGISTRY}/dbld-$*:latest; \
IMAGE_ID=$$($(DOCKER) images -q $$IMAGE | head -1); \
WATCHED_FILES="dbld packaging/rhel/syslog-ng.spec packaging/debian/control"; \
if [ "$$IMAGE_ID" = "" ]; then \
$(DBLD_RULES) pull-image-$*; \
fi; \
IMAGE_ID=$$($(DOCKER) images -q $$IMAGE | head -1); \
if [ "$$IMAGE_ID" != "" ]; then \
image_commit=$$($(DOCKER) inspect --format '{{ index .Config.Labels "COMMIT"}}' $$IMAGE_ID); \
dbld_changes=$$( \
($(GIT) cat-file -e $${image_commit:-NO_COMMIT_LABEL_IN_DBLD_IMAGE}^{commit} \
&& $(GIT) diff $${image_commit} -- $${WATCHED_FILES} | wc -l \
|| echo 1) \
); \
else \
dbld_changes=1; \
fi; \
if [ "$${dbld_changes}" -gt 0 ]; then \
echo "Triggering rebuild of $$IMAGE, as dbld directory changed since the build of the image, or the " \
"cached image has no COMMIT label, or couldn't fetch from dockerhub or couldn't find git" && \
$(DBLD_RULES) image-$*; \
else \
echo "DBLD version in image matches, not initiating rebuild..."; \
fi
exec: exec-$(DEFAULT_IMAGE)
exec: EXEC_COMMAND=echo Specify EXEC_COMMAND to do something sensible here
exec-%: setup
@container=`$(DOCKER) ps | grep dbld-$* | head -1 | cut -d ' ' -f1`; \
$(DOCKER) exec ${DOCKER_INTERACTIVE} $$container $(EXEC_COMMAND)
login: login-$(DEFAULT_IMAGE)
login-%: EXEC_COMMAND=sudo -u $(shell whoami) /dbld/shell
login-%: exec-%
@true
root-login: root-login-$(DEFAULT_IMAGE)
root-login-%: EXEC_COMMAND=bash
root-login-%: exec-%
@true
setup:
@mkdir -p dbld/build || true
@mkdir -p dbld/install || true
@mkdir -p dbld/release || true
list-builder-images:
@echo ${BUILDER_IMAGES}
|