1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931
|
#!/usr/bin/env python3
# -----------------------------------------------------------------------------
# Copyright (c) 2014-2023, PyInstaller Development Team.
#
# Distributed under the terms of the GNU General Public License (version 2
# or later) with exception for distributing the bootloader.
#
# The full license is in the file COPYING.txt, distributed with this software.
#
# SPDX-License-Identifier: (GPL-2.0-or-later WITH Bootloader-exception)
# -----------------------------------------------------------------------------
"""
Bootloader building script.
"""
import os
import platform
import sys
import re
import sysconfig
from waflib.Configure import conf
from waflib import Logs, Utils, Options
from waflib.Build import BuildContext, InstallContext
# waf does not work unless it is ran from in this folder, so it is safe to assume the current working directory here.
sys.path.insert(0, os.path.realpath("../PyInstaller/"))
import _shared_with_waf # noqa: E402
# The following two variables are used by the target "waf dist"
VERSION = 'nodist'
APPNAME = 'PyInstallerBootloader'
# These variables are mandatory ('/' are converted automatically)
top = '.'
out = 'build'
# Build variants of bootloader.
# PyInstaller provides debug/release bootloaders and console/windowed variants. Each variant has a different exe name.
variants = {
'debug': 'run_d',
'debugw': 'runw_d',
'release': 'run',
'releasew': 'runw',
}
# PyInstaller only knows platform.system(), so we need to map waf's DEST_OS to these values.
DESTOS_TO_SYSTEM = {
'linux': 'Linux',
'freebsd': 'FreeBSD',
'openbsd': 'OpenBSD',
'win32': 'Windows',
'darwin': 'Darwin',
'sunos': platform.system(), # FIXME: inhibits cross-compile
'hpux': 'HP-UX',
'aix': 'AIX',
'cygwin': 'Cygwin',
}
# Map from platform.system() to waf's DEST_OS
SYSTEM_TO_BUILDOS = {
'Linux': 'linux',
'FreeBSD': 'freebsd',
'Windows': 'win32',
'Darwin': 'darwin',
'Solaris': 'sunos',
'SunOS': 'sunos',
'HP-UX': 'hpux',
'AIX': 'aix',
'Cygwin': 'cygwin',
}
# waf's name of the system we are building on
if sys.platform == 'cygwin':
# Cygwin needs special handling, because platform.system() contains identifiers such as MSYS_NT-10.0-19042 and
# CYGWIN_NT-10.0-19042 that do not fit PyInstaller's OS naming scheme, nor waf's DEST_OS, which is 'cygwin'.
BUILD_OS = SYSTEM_TO_BUILDOS.get('Cygwin')
else:
BUILD_OS = SYSTEM_TO_BUILDOS.get(platform.system(), platform.system())
is_cross = None
def machine(ctx):
"""
Determine path to bootloader based on C compiler target.
"""
return _shared_with_waf._pyi_machine(ctx.env.DEST_CPU, DESTOS_TO_SYSTEM[ctx.env.DEST_OS])
def is_msvc_target(ctx):
"""
clang may be compiled to use either MSVC's headers or mingw's. If it's the former, it inherits some of MSVC's
quirks.
"""
if ctx.env.CC_NAME == 'clang':
from subprocess import run, PIPE
spec = run(ctx.env.CC + ["-v"], stderr=PIPE).stderr
return re.search(rb'Target: .*msvc', spec)
return ctx.env.CC_NAME == 'msvc'
@conf
def is_musl(ctx):
"""Return true if the C compiler targets Linux using musl libc. Always false on non-Linux."""
if ctx.env.DEST_OS != 'linux':
return False
from subprocess import run, PIPE
spec = run(ctx.env.CC + ["-v"], stderr=PIPE).stderr
return re.search(rb'--target=\S+-musl', spec) or re.search(rb'Target: .*musl', spec)
def options(ctx):
ctx.load('compiler_c')
ctx.add_option(
'--debug',
action='store_true',
help='Include debugging info for GDB.',
default=False,
dest='debug',
)
ctx.add_option(
'--clang',
action='store_true',
help='Try to find clang C compiler instead of gcc.',
default=False,
dest='clang',
)
ctx.add_option(
'--gcc',
action='store_true',
help='Try to find GNU C compiler.',
default=False,
dest='gcc',
)
ctx.add_option(
'--target-arch',
action='store',
help='Target architecture format (32bit, 64bit). This option allows to build 32-bit bootloader with a 64-bit '
'compiler and a 64-bit Python.',
default=None,
choices=('32bit', '64bit', '64bit-arm'),
dest='target_arch',
)
ctx.add_option(
'--static-zlib',
action='store_true',
help='Statically compile zlib into the bootloaders rather than dynamically linking to the system-wide copy. '
'This is always done on Windows.',
default=False,
)
ctx.add_option(
'--tests',
action='store_true',
help='Enable cmocka-based tests if cmocka library is found. The tests are disabled by default.',
default=False,
dest='enable_tests',
)
ctx.add_option(
'--no-tests',
action='store_false',
help='Disable cmocka-based tests, even if cmocka library is found. The tests are disabled by default.',
default=False,
dest='enable_tests',
)
grp = ctx.add_option_group('macOS-specific options', 'These options have effect only on macOS.')
grp.add_option(
'--universal2',
action='store_true',
help='When building for 64-bit platform, build universal2 fat binary (x86_64, arm64). This is the default.',
default=None,
dest='macos_universal2',
)
grp.add_option(
'--no-universal2',
action='store_false',
help="When building for 64-bit platform, build a thin binary. Allows building with older toolchains that do "
"not support universal2 binaries. The resultant architecture is determined by the compiler's default, which "
"is usually to build a native executable. This may be overridden by setting the CC environment variable, e.g., "
"CC='clang -arch=arm64' python waf all",
default=None,
dest='macos_universal2',
)
def check_sizeof_pointer(ctx):
def check(type, expected):
# test code taken from autoconf resp. Scons: this is a pretty clever hack to find that a type is of a given
# size using only compilation. This speeds things up quite a bit compared to straightforward code actually
# running the code. Plus: This works cross :-)
fragment = '''
int main() {
static int test_array[1 - 2 * !(sizeof(%s) == %d)];
test_array[0] = 0;
return 0;
}''' % (type, expected)
return ctx.check_cc(fragment=fragment, execute=False, mandatory=False)
ctx.start_msg("Checking size of pointer")
for size in (4, 8):
if check("void *", size):
break
else:
ctx.end_msg(False)
ctx.fatal(
"Could not determine pointer size. Use `--target-arch' to manually set the pointer size (32bit or 64bit)."
)
ctx.end_msg(size)
return size
@conf
def detect_arch(ctx):
"""
Handle --target-arch options, or use the same architecture as the compiler.
"""
try:
system = DESTOS_TO_SYSTEM[ctx.env.DEST_OS]
except KeyError:
ctx.fatal('Unrecognized target system: %s' % ctx.env.DEST_OS)
# Get arch values either from CLI or detect it.
if ctx.options.target_arch:
arch = ctx.options.target_arch
ctx.msg('Platform', '%s-%s manually chosen' % (system, arch))
ctx.env.ARCH_FLAGS_REQUIRED = True
if arch == '64bit-arm':
arch = '64bit'
ctx.env.DEST_CPU = 'arm64'
else:
# PyInstaller uses the result of platform.architecture() to determine the bits and this is testing the pointer
# size (via module struct). We do the same here.
arch = "%sbit" % (8 * check_sizeof_pointer(ctx))
_machine = machine(ctx)
if _machine is not None:
plat = '%s-%s-%s detected based on compiler' % (system, arch, _machine)
else:
plat = '%s-%s detected based on compiler' % (system, arch)
ctx.msg('Platform', plat)
ctx.env.ARCH_FLAGS_REQUIRED = False
if arch not in ('32bit', '64bit'):
ctx.fatal('Unrecognized target architecture: %s' % arch)
# Pass return values as environment variables.
ctx.env.PYI_ARCH = arch # '32bit' or '64bit'
ctx.env.PYI_SYSTEM = system
@conf
def set_arch_flags(ctx):
"""
Set proper architecture flag (32-bit or 64-bit), cflags for compiler, and CPU target for compiler.
"""
def check_arch_cflag(cflag32, cflag64):
cflag = cflag32 if ctx.env.PYI_ARCH == '32bit' else cflag64
# features='c' -> only compile, do not link
if ctx.check_cc(cflags=cflag, features='c', mandatory=ctx.env.ARCH_FLAGS_REQUIRED):
ctx.env.append_value('CFLAGS', cflag)
if ctx.check_cc(linkflags=cflag, mandatory=ctx.env.ARCH_FLAGS_REQUIRED):
ctx.env.append_value('LINKFLAGS', cflag)
if ctx.env.DEST_OS == 'win32' and ctx.env.CC_NAME == 'msvc':
# Set msvc linkflags based on architecture.
if machine(ctx) == 'arm':
ctx.env.append_value('LINKFLAGS', '/MACHINE:ARM64')
elif ctx.env.PYI_ARCH == '32bit':
ctx.env.append_value('LINKFLAGS', '/MACHINE:X86')
# Set LARGE_ADDRESS_AWARE_FLAG to True. On Windows, this allows 32-bit apps to use 4GB of memory.
ctx.env.append_value('LINKFLAGS', '/LARGEADDRESSAWARE')
elif ctx.env.PYI_ARCH == '64bit':
ctx.env.append_value('LINKFLAGS', '/MACHINE:X64')
ctx.env['MSVC_TARGETS'] = _msvc_target(ctx)
# Enable 64-bit porting warnings and other warnings.
ctx.env.append_value('CFLAGS', '/W3')
# Treat compiler warnings as errors.
ctx.env.append_value('CFLAGS', '/WX')
# Disable warnings about unrecognized pragmas.
ctx.env.append_value('CFLAGS', '/wd4068')
# Disable warnings about deprecated POSIX function names.
ctx.env.append_value('CFLAGS', '/D_CRT_NONSTDC_NO_WARNINGS')
# Disable warnings about unsafe CRT functions.
ctx.env.append_value('CFLAGS', '/D_CRT_SECURE_NO_WARNINGS')
# We use SEH exceptions in winmain.c; make sure they are activated.
ctx.env.append_value('CFLAGS', '/EHa')
# Set the PE checksum on resulting binary.
ctx.env.append_value('LINKFLAGS', '/RELEASE')
# Treat linker warnings as errors
ctx.env.append_value('LINKFLAGS', '/WX')
# Ensure proper architecture flags on macOS.
elif ctx.env.DEST_OS == 'darwin':
# Default compiler on macOS is Clang, which does not have '-m32' and '-m64' flags.
if ctx.env.PYI_ARCH == '32bit':
mac_arch = ['-arch', 'i386']
else:
UNIVERSAL2_FLAGS = ['-arch', 'x86_64', '-arch', 'arm64']
if ctx.options.macos_universal2 in (None, True):
# Check for universal2 support if set to auto-detect (neither --universal2 nor --no-universal2 is
# provided), or if universal2 is explicitly enabled.
supported = ctx.check_cc(
cflags=UNIVERSAL2_FLAGS,
linkflags=UNIVERSAL2_FLAGS,
features='c cprogram',
msg='Checking for universal2 support',
mandatory=ctx.options.macos_universal2
)
ctx.options.macos_universal2 = supported
if ctx.options.macos_universal2:
mac_arch = UNIVERSAL2_FLAGS
else:
# Default to whatever the compiler is configured to build.
mac_arch = []
ctx.env.append_value('CFLAGS', mac_arch)
ctx.env.append_value('LINKFLAGS', mac_arch)
# AIX specific flags
elif ctx.env.DEST_OS == 'aix':
if ctx.env.CC_NAME == 'gcc':
check_arch_cflag('-maix32', '-maix64')
else:
# We are using AIX/xlc compiler
check_arch_cflag('-q32', '-q64')
elif ctx.env.DEST_OS == 'sunos':
if ctx.env.CC_NAME == 'gcc':
check_arch_cflag('-m32', '-m64')
else:
# We use SUNWpro C compiler
check_arch_cflag('-xarch=generic', '-xarch=v9')
elif ctx.env.DEST_OS == 'hpux':
if ctx.env.CC_NAME == 'gcc':
check_arch_cflag('-milp32', '-mlp64')
else:
# We use xlc compiler
pass
# Other compiler - not msvc.
else:
if machine(ctx) == 'sw_64':
# The gcc has no '-m64' option under sw64 machine, but the __x86_64__ macro needs to be defined.
ctx.env.append_value('CCDEFINES', '__x86_64__')
# This ensures proper compilation with 64-bit gcc and 32-bit Python or vice versa or with manually
# chosen --target-arch. Option -m32/-m64 has to be passed to cflags and linkflages.
else:
check_arch_cflag('-m32', '-m64')
if ctx.env.PYI_ARCH == '32bit' and ctx.env.DEST_OS == 'win32':
# Set LARGE_ADDRESS_AWARE_FLAG to True. On Windows, this allows 32-bit apps to use 4GB of memory.
# TODO: verify if this option being as default might cause any side effects.
ctx.env.append_value('LINKFLAGS', '-Wl,--large-address-aware')
# A bare minimum zlib program. Testing zlib-dev availability requires explicitly using libz-provided symbols instead of
# just compiling 'include <zlib.h>' with `-lz`, because while zlib's development headers may be installed, the library
# itself may not be linkable (e.g., OpenWRT strips their binaries using `sstrip`).
ZLIB_TEST_FRAGMENT = r"""
#include <stdio.h>
#include <zlib.h>
int main ()
{
printf("Zlib: %s\n", zlibVersion());
return 0;
}
"""
def _msvc_target(ctx):
if getattr(ctx.options, "msvc_targets", False):
return re.findall(r"[^, ]+", ctx.options.msvc_targets)
try:
_msvc_names = {"AMD64": "amd64", "x86": "x86", "ARM64": "arm64"}
host = _msvc_names[os.environ.get("PROCESSOR_ARCHITECTURE", platform.machine())]
except KeyError:
return []
if ctx.options.target_arch == '32bit':
target = 'x86'
elif ctx.options.target_arch == '64bit':
target = 'x64'
elif ctx.options.target_arch == '64bit-arm':
target = 'arm64'
else:
target = "x64" if host == "amd64" else host
if host == 'x86':
if target == 'x86':
return ['x86']
elif target == 'x64':
return ['x86_amd64']
else: # arm64
return ['x86_arm64']
elif host == 'amd64':
if target == 'x86':
return ['amd64_x86', 'x86']
elif target == 'x64':
return ['x64', 'x86_amd64']
else: # arm64
return ['amd64_arm64', 'x86_arm64']
else: # arm64
if target == 'x86':
return ['arm64_x86', 'x86', 'amd64_x86']
elif target == 'x64':
return ['arm64_amd64', 'x64', 'x86_amd64']
else: # arm64
return ['arm64']
def configure(ctx):
ctx.msg('Python Version', sys.version.replace(os.linesep, ''))
# For MSVC the target arch must have already been set when the compiler is searched.
ctx.env['MSVC_TARGETS'] = _msvc_target(ctx)
ctx.msg('MSVC target(s)', ctx.env['MSVC_TARGETS'])
# ** C compiler **
# Allow to use Clang if preferred.
if ctx.options.clang:
ctx.load('clang')
# Allow to use gcc if preferred.
elif ctx.options.gcc:
ctx.load('gcc')
else:
ctx.load('compiler_c') # Any available C compiler.
global is_cross
is_cross = (BUILD_OS != ctx.env.DEST_OS)
if is_cross:
ctx.msg('System', 'Assuming cross-compilation for %s' % DESTOS_TO_SYSTEM[ctx.env.DEST_OS])
if ctx.env.DEST_OS in ('hpux', 'sunos'):
# For SunOS and HP-UX we determine some settings from Python's sysconfig. For cross-compiling, somebody
# needs to implement options to overwrite these values as they may be wrong.
# For SunOS/Solaris mapping DEST_OS to system is not yet known.
ctx.fatal(
'Cross-compiling for target %s is not yet supported. If you want this feature, please help '
'implementing it. See the wscript file for details.' % ctx.env.DEST_OS
)
if ctx.env.DEST_OS == 'darwin':
# macOS 10.13 is the oldest version supported by Apple. According to macOS docs, setting the
# MACOSX_DEPLOYMENT_TARGET environment variable is equivalent to the following gcc option:
# -mmacosx-version-min=10.13
#
# MACOSX_DEPLOYMENT_TARGET must be set before the compiler toolchain is used for the first time (i.e., the
# check_sizeof_pointer() call made by the detect_arch() call below), otherwise it will be automatically
# set to the toolchain's default value.
if not os.environ.get('MACOSX_DEPLOYMENT_TARGET'):
ctx.msg('MacOS deployment target', 'Setting to 10.13')
os.environ['MACOSX_DEPLOYMENT_TARGET'] = '10.13'
else:
ctx.msg('MacOS deployment target', 'Already set - %s' % (os.environ.get('MACOSX_DEPLOYMENT_TARGET')))
# Detect architecture after completing compiler search
ctx.detect_arch()
# Set proper architecture and CPU for C compiler
ctx.set_arch_flags()
# ** Other Tools **
if ctx.env.DEST_OS == 'win32':
# Do not embed manifest file when using MSVC (Visual Studio). Manifest file will be added in the phase of
# packaging python application by PyInstaller.
ctx.env.MSVC_MANIFEST = False
# ** C Compiler optimizations **
# TODO Set proper optimization flags for MSVC (Visual Studio).
if ctx.options.debug:
if ctx.env.DEST_OS == 'win32' and ctx.env.CC_NAME == 'msvc':
# Include information for debugging in MSVC/msdebug
ctx.env.append_value('CFLAGS', '/Z7')
ctx.env.append_value('CFLAGS', '/Od')
ctx.env.append_value('LINKFLAGS', '/DEBUG')
else:
# Include gcc debugging information for debugging in GDB.
ctx.env.append_value('CFLAGS', '-g')
else:
if ctx.env.DEST_OS != 'sunos':
ctx.env.append_value('CFLAGS', '-O2')
else:
# Solaris SUN CC doesn't support '-O2' flag
ctx.env.append_value('CFLAGS', '-O')
if ctx.env.CC_NAME == 'gcc':
# These flags are specific to gcc!
# Turn on all warnings to improve code quality and avoid errors. Unused variables and unused functions are still
# accepted to avoid even more conditional code. If you are ever tempted to change this, review the commit
# history of this place first.
ctx.env.append_value(
'CFLAGS', [
'-Wall',
'-Werror',
'-Wno-error=unused-variable',
'-Wno-error=unused-function',
]
)
# -Wno-error=unused-but-set-variable is not available on old gcc versions (e.g., 4.4.3)
if ctx.check_cc(cflags='-Wno-error=unused-but-set-variable', execute=False, mandatory=False):
ctx.env.append_value('CFLAGS', ['-Wno-error=unused-but-set-variable'])
elif ctx.env.CC_NAME == 'clang':
# For clang, also enable extra warnings and treat them as errors. Similarly to gcc, do not treat warnings about
# unused variables and unused functions as errors.
ctx.env.append_value(
'CFLAGS', [
'-Wall',
'-Werror',
'-Wno-error=unused-variable',
'-Wno-error=unused-function',
]
)
# ** Defines, includes **
if not ctx.env.DEST_OS == 'win32':
# Defines common for Unix and Unix-like platforms. For details see:
# http://man.he.net/man7/feature_test_macros
ctx.env.append_value('DEFINES', '_REENTRANT')
# mkdtemp() is available only if _BSD_SOURCE is defined.
ctx.env.append_value('DEFINES', '_BSD_SOURCE')
if ctx.env.DEST_OS == 'linux':
# GCC 5.x complains about _BSD_SOURCE under Linux:
# _BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE
ctx.env.append_value('DEFINES', '_DEFAULT_SOURCE')
# TODO: What other platforms support _FORTIFY_SOURCE macro? macOS?
# TODO: macOS's CLang appears to support this macro as well. See:
# https://marc.info/?l=cfe-dev&m=122032133830183
# For security, enable the _FORTIFY_SOURCE macro detecting buffer overflows in various string and memory
# manipulation functions.
if ctx.options.debug:
ctx.env.append_value('CFLAGS', '-U_FORTIFY_SOURCE')
elif ctx.env.CC_NAME == 'gcc':
# Undefine this macro if already defined by default to avoid "macro redefinition" errors.
ctx.env.append_value('CFLAGS', '-U_FORTIFY_SOURCE')
# Define this macro.
ctx.env.append_value('DEFINES', '_FORTIFY_SOURCE=2')
# On macOS, mkdtemp() is available only with _DARWIN_C_SOURCE.
elif ctx.env.DEST_OS == 'darwin':
ctx.env.append_value('DEFINES', '_DARWIN_C_SOURCE')
# Determine which semaphore API to use for syncing onefile parent and child process on non-Windows platforms;
# this is required to ensure consistent signal forwarding behavior. We prefer to use POSIX semaphores; however,
# they are unavailable on macOS, where we need to fall back to old SysV semaphore API.
semaphore_api = None
if not ctx.env.DEST_OS == 'darwin':
if ctx.check(header_name='semaphore.h', mandatory=False):
semaphore_api = 'posix'
if not semaphore_api:
if ctx.check(header_name='sys/sem.h', mandatory=False):
semaphore_api = 'sysv'
if not semaphore_api:
semaphore_api = 'none' # For the message
ctx.msg("Using semaphore API", semaphore_api)
if semaphore_api == 'sysv':
ctx.env.append_value('DEFINES', 'PYI_USE_SYSV_SEMAPHORE')
elif semaphore_api == 'posix':
ctx.env.append_value('DEFINES', 'PYI_USE_POSIX_SEMAPHORE')
if ctx.env.DEST_OS == 'win32':
ctx.env.append_value('DEFINES', 'WIN32')
ctx.env.append_value('CPPPATH', '../zlib')
# Set minimum feature level for Windows headers to Windows 7.
# As per MSDN: "If you define NTDDI_VERSION, you must also define _WIN32_WINNT."
# https://docs.microsoft.com/en-us/windows/win32/winprog/using-the-windows-headers#macros-for-conditional-declarations
ctx.env.append_value('DEFINES', 'NTDDI_VERSION=0x06010000')
ctx.env.append_value('DEFINES', '_WIN32_WINNT=0x0601')
elif ctx.env.DEST_OS == 'sunos':
ctx.env.append_value('DEFINES', 'SUNOS')
if ctx.env.CC_NAME == 'gcc':
# On Solaris using gcc, the linker options for shared and static libraries are slightly different from
# other platforms.
ctx.env['SHLIB_MARKER'] = '-Wl,-Bdynamic'
ctx.env['STLIB_MARKER'] = '-Wl,-Bstatic'
# On Solaris using gcc, the compiler needs to be gnu99
ctx.env.append_value('CFLAGS', '-std=gnu99')
elif ctx.env.DEST_OS == 'aix':
ctx.env.append_value('DEFINES', 'AIX')
# On AIX some APIs are restricted if _ALL_SOURCE is not defined. In the case of PyInstaller, we need the AIX
# specific flag RTLD_MEMBER for dlopen(), which is used to load a shared object from a library archive.
# We need to load the Python library like this:
# dlopen("libpython2.7.a(libpython2.7.so)", RTLD_MEMBER)
ctx.env.append_value('DEFINES', '_ALL_SOURCE')
# On AIX using gcc, the linker options for shared and static libraries are slightly different from
# other platforms.
ctx.env['SHLIB_MARKER'] = '-Wl,-bdynamic'
ctx.env['STLIB_MARKER'] = '-Wl,-bstatic'
elif ctx.env.DEST_OS == 'hpux':
ctx.env.append_value('DEFINES', 'HPUX')
if ctx.env.CC_NAME == 'gcc':
if ctx.env.PYI_ARCH == '32bit':
ctx.env.append_value('LIBPATH', '/usr/local/lib/hpux32')
ctx.env.append_value('STATICLIBPATH', '/usr/local/lib/hpux32')
else:
ctx.env.append_value('LIBPATH', '/usr/local/lib/hpux64')
ctx.env.append_value('STATICLIBPATH', '/usr/local/lib/hpux64')
# ** Libraries **
if ctx.env.DEST_OS == 'win32':
if ctx.env.CC_NAME == 'msvc':
ctx.check_libs_msvc('user32 comctl32 kernel32 advapi32 gdi32', mandatory=True)
else:
ctx.check_cc(lib='user32', mandatory=True)
ctx.check_cc(lib='comctl32', mandatory=True)
ctx.check_cc(lib='kernel32', mandatory=True)
ctx.check_cc(lib='advapi32', mandatory=True)
ctx.check_cc(lib='gdi32', mandatory=True)
else:
# On most platforms, the libdl symbols are moved to libc. The POSIX
# standard states that libdl should always be linkable against, even if
# it's empty, but not all provide appropriate libdl.a stub archives.
# Treat it as optional.
ctx.check_cc(lib='dl', mandatory=False)
# libdl to be thread-safe requires libpthread! Assume it must be available if libdl is available.
ctx.check_cc(lib='pthread', mandatory=bool(ctx.env.LIB_DL))
if ctx.env.DEST_OS == 'freebsd' and sysconfig.get_config_var('HAVE_PTHREAD_H'):
# On FreeBSD if python has threads: libthr needs to be loaded in the main process, so the bootloader needs
# to be link to thr.
ctx.check_cc(lib='thr', mandatory=True)
elif ctx.env.DEST_OS == 'hpux' and sysconfig.get_config_var('HAVE_PTHREAD_H'):
ctx.check_cc(lib='pthread', mandatory=True)
ctx.check_cc(lib='m', mandatory=True)
# Opting out of dynamically linked zlib can be done either with the --static-zlib option or with the
# PYI_STATIC_ZLIB=1 environment variable.
static_zlib = bool(int(os.environ.get("PYI_STATIC_ZLIB", '0'))) or ctx.options.static_zlib
# On AIX, use static build of bundled zlib.
if ctx.env.DEST_OS == 'aix':
static_zlib = True
if static_zlib:
# This serves as a signal to later on when the C code gets compiled.
ctx.env.LIB_Z = None
else:
ctx.check_cc(lib='z', mandatory=False, uselib_store='Z', fragment=ZLIB_TEST_FRAGMENT)
if not ctx.env.LIB_Z:
ctx.fatal(
"The zlib development package is either missing or the shared library cannot be linked against. "
"For security (and marginally better filesize), you should install the zlib-dev or zlib-devel "
"packages with your system package manager, and try again. If you cannot do this (for example, "
"distributions such as OpenWRT use sstrip on libraries, making linking impossible), then either "
"use the --static-zlib option or set the PYI_STATIC_ZLIB=1 environment variable. If you are "
"installing directly with pip, then use the environment variable."
)
ctx.recurse("tests")
# ** Functions and headers **
# Check for presence of stdbool.h
ctx.check(header_name='stdbool.h', mandatory=False)
# The old ``function_name`` parameter to ``check_cc`` is no longer supported. This code is based on old waf
# source at
# https://gitlab.com/ita1024/waf/commit/62fe305d04ed37b1be1a3327a74b2fee6c458634#255b2344e5268e6a34bedd2f8c4680798344fec7.
SNIP_FUNCTION = '''
#include <%s>
int main(int argc, char **argv) {
void (*p)();
(void)argc; (void)argv;
p=(void(*)())(%s);
return !p;
}
'''
# OS support for these functions varies.
for header, function_name in (
('stdlib.h', 'unsetenv'),
# Most systems have `mkdtemp` defined in `stdlib.h`; except for macOS, where it is defined in `unistd.h`.
('unistd.h' if ctx.env.DEST_OS == 'darwin' else 'stdlib.h', 'mkdtemp'),
('libgen.h', 'dirname'),
('libgen.h', 'basename'),
('wchar.h', 'wcsdup')
):
ctx.check(
fragment=SNIP_FUNCTION % (header, function_name),
mandatory=False,
define_name=ctx.have_define(function_name),
msg='Checking for function %s' % function_name
)
# ** CFLAGS **
if ctx.env.DEST_OS == 'win32':
if ctx.env.CC_NAME == 'msvc':
# Use Unicode entry point wmain/wWinMain and wchar_t WinAPI
ctx.env.append_value('CFLAGS', '-DUNICODE')
ctx.env.append_value('CFLAGS', '-D_UNICODE')
# Specify CONSOLE subsystem, without major/minor version. At the time of writing, the compiler's default
# 6.00 (x86, x64) and 6.02 (ARM) should be fine for our purposes.
ctx.env.append_value('LINKFLAGS', '/SUBSYSTEM:CONSOLE')
# Enable support for Control Flow Guard
# https://docs.microsoft.com/en-us/windows/win32/secbp/control-flow-guard
ctx.env.append_value('CFLAGS', '/guard:cf')
ctx.env.append_value('LINKFLAGS', '/guard:cf')
# Increase the executable's stack size from the default 1 MiB (0x100000) to 2 MB (0x1E8480) to match
# the stack size under the Python interpreter.
# https://github.com/python/cpython/blob/a9e0b2b49374df91c40fe409508cfcdc6332450e/PCbuild/python.vcxproj#L97
ctx.env.append_value('LINKFLAGS', '/stack:2000000')
else:
# Use Visual C++ compatible alignment
ctx.env.append_value('CFLAGS', '-mms-bitfields')
# Define UNICODE and _UNICODE for wchar_t WinAPI
ctx.env.append_value('CFLAGS', '-municode')
# NOTE: the following two defines are added on the off-chance
# that we are building with clang + MSVC. For pure MSVC, they
# are specified in `set_arch_flags()`.
# Disable warnings about deprecated POSIX function names.
ctx.env.append_value('CFLAGS', '-D_CRT_NONSTDC_NO_WARNINGS')
# Disable warnings about unsafe CRT functions.
ctx.env.append_value('CFLAGS', '-D_CRT_SECURE_NO_WARNINGS')
# Use Unicode entry point wmain/wWinMain
ctx.env.append_value('LINKFLAGS', '-municode')
# Set the executable's stack size to 2 MB (0x1E8480) to match the stack size under the Python interpreter.
# The MSYS2/MINGW64 gcc toolchain seems to use 2 MiB (0x200000) stack by default, so we are slightly
# decreasing the stack size here, in order to keep it synchronized with the python itself and with the
# MSVC-compiled bootloaders.
if not is_msvc_target(ctx):
ctx.env.append_value('LINKFLAGS', '-Wl,--stack,2000000')
# On linux link only with needed libraries.
# On some platforms (macOS, Solaris, AIX), -Wl,--as-needed is detected as supported during configuration,
# but fails during build. So use it only with linux.
if ctx.env.DEST_OS == 'linux' and ctx.check_cc(linkflags='-Wl,--as-needed', mandatory=False):
ctx.env.append_value('LINKFLAGS', '-Wl,--as-needed')
# When not building with MSVC, strip the bootloader executables to reduce their size.
if not is_msvc_target(ctx):
ctx.load('strip', tooldir='tools') # Load strip feature/tool from ./tools directory.
def windowed(name, baseenv):
"""Setup windowed environment based on `baseenv`."""
ctx.setenv(name, baseenv) # Inherit from `baseenv`.
ctx.env.append_value('DEFINES', 'WINDOWED')
if ctx.env.DEST_OS == 'win32':
if ctx.env.CC_NAME != 'msvc':
# For MinGW disable console window on Windows - MinGW option.
# This option is for linker, and thus needs to be specified under
# LINKFLAGS.
#
# While gcc does not seem to mind having it under CFLAGS
# as well, clang seems to complain about it:
# `clang: warning: argument unused during compilation:
# '-mwindows' [-Wunused-command-line-argument]`. Recent
# versions, when coupled with MSVC linker, raise an error:
# `clang: error: unsupported option '-mwindows' for target
# 'x86_64-pc-windows-msvc'`.
#
# NOTE: under both contemporary gcc and clang, the
# subsystem seems automatically inferred from available
# entry-point, so explicitly specifiying it here is likely
# redundant.
ctx.env.append_value('LINKFLAGS', '-mwindows')
else:
# Remove any /SUBSYSTEM flag already present in LINKFLAGS
_link_flags = ctx.env._get_list_value_for_modification('LINKFLAGS')
_subsystem = [x for x in _link_flags if x.startswith('/SUBSYSTEM:')]
for parameter in _subsystem:
_link_flags.remove(parameter)
# Specify WINDOWS subsystem, without major/minor version. At the time of writing, the compiler's
# default 6.00 (x86, x64) and 6.02 (ARM) should be fine for our purposes.
ctx.env.append_value('LINKFLAGS', '/SUBSYSTEM:WINDOWS')
elif ctx.env.DEST_OS == 'darwin':
# To support catching AppleEvents and running as ordinary macOS GUI app, we have to link against the
# Carbon framework. This linkage only needs to be there for the windowed bootloaders.
ctx.env.append_value('LINKFLAGS', '-framework')
ctx.env.append_value('LINKFLAGS', 'Carbon')
# TODO Do we need to link with this framework?
# conf.env.append_value('LINKFLAGS', '-framework')
# conf.env.append_value('LINKFLAGS', 'ApplicationServices')
# ** DEBUG and RELEASE environments **
basic_env = ctx.env
# * Setup DEBUG environment *
ctx.setenv('debug', basic_env) # Ensure env contains shared values.
debug_env = ctx.env
# This defines enable verbose console output of the bootloader.
ctx.env.append_value('DEFINES', ['LAUNCH_DEBUG'])
ctx.env.append_value('DEFINES', 'NDEBUG')
# * Setup windowed DEBUG environment *
windowed('debugw', debug_env)
# * Setup RELEASE environment *
ctx.setenv('release', basic_env) # Ensure env contains shared values.
release_env = ctx.env
ctx.env.append_value('DEFINES', 'NDEBUG')
# * Setup windowed RELEASE environment *
windowed('releasew', release_env)
def build(ctx):
if not ctx.variant:
ctx.fatal('Call "python waf all" to compile all bootloaders.')
exe_name = variants[ctx.variant]
install_path = os.path.join(os.getcwd(), '../PyInstaller/bootloader', ctx.env.PYI_SYSTEM + "-" + ctx.env.PYI_ARCH)
install_path = os.path.normpath(install_path)
if machine(ctx) and ctx.env.DEST_OS != 'darwin':
install_path += '-' + machine(ctx)
if ctx.is_musl():
install_path += "-musl"
if not ctx.env.LIB_Z:
# If the operating system does not provide zlib, build our own. The configure phase defines whether or not zlib
# is mandatory for the given platform.
ctx.objects(source=ctx.path.ant_glob('zlib/*.c'), target='STATIC_ZLIB', includes='zlib')
# Unless we are building with MSVC, strip bootloader executables to make them smaller.
features = '' if is_msvc_target(ctx) else 'strip'
ctx.objects(source=ctx.path.ant_glob('src/*.c', excl="src/main.c"), includes='src windows zlib', target="OBJECTS")
ctx.env.link_with_dynlibs = []
ctx.env.link_with_staticlibs = []
if ctx.env.DEST_OS == 'win32':
# On Windows we need to link library zlib statically.
ctx.program(
source=['src/main.c'],
target=exe_name,
install_path=install_path,
use='OBJECTS USER32 COMCTL32 KERNEL32 ADVAPI32 GDI32 Z STATIC_ZLIB',
includes='src windows zlib',
features=features
)
else:
# Linux, Darwin (macOS), ...
# Only the libs found will actually be used, so it's safe to list all here. The decision if a lib is required
# for a specific platform is made in the configure phase.
libs = [
'DL',
'M', # math
'Z', # zlib
'PTHREAD', # important! needs for libdl to be thread-safe
'THR', # may be used on FreBSD
]
staticlibs = []
ctx.env.link_with_dynlibs = libs
ctx.env.link_with_staticlibs = staticlibs
ctx.program(
source='src/main.c',
target=exe_name,
includes='src',
use=libs + ["OBJECTS", "STATIC_ZLIB"],
stlib=staticlibs,
install_path=install_path,
features=features
)
# This warning is deliberately at the end of the build, in the hopes that it will be more visible amongst all the
# other stuff written to stdout.
if machine(ctx) and machine(ctx) == "unknown":
Logs.log.warning(
"The target architecture reported by the compiler '%s' was not recognised and defaulted to 'unknown'. You "
"should verify that `PyInstaller.compat.machine` is also 'unknown' on the target machine. If it is not, "
"please add '%s' to `PyInstaller._shared_with_waf._pyi_machine()` and submit a pull request. Without this "
"match, PyInstaller will not be able to find the bootloaders you just built!" %
(ctx.env.DEST_CPU, ctx.env.DEST_CPU)
)
ctx.recurse("tests")
class make_all(BuildContext):
"""
Do build and install in one step.
"""
cmd = 'make_all'
def execute_build(ctx):
Options.commands = ['build_debug', 'build_release']
# On Windows and macOS we also need console/windowed bootloaders. On other platforms they make no sense.
if ctx.env.DEST_OS in ('win32', 'darwin'):
Options.commands += ['build_debugw', 'build_releasew']
# Install bootloaders.
Options.commands += ['install_debug', 'install_release']
if ctx.env.DEST_OS in ('win32', 'darwin'):
Options.commands += ['install_debugw', 'install_releasew']
def all(ctx):
"""
Do configure, build and install in one step.
"""
# `all` is run prior to `configure`, thus it does not get a build context. Therefore another command `make_all` is
# required, which gets the build context, and can make decisions based on the outcome of `configure`.
Options.commands = ['distclean', 'configure', 'make_all']
# Set up building several variants of bootloader.
for x in variants:
class BootloaderContext(BuildContext):
cmd = 'build' + '_' + x
variant = x
class BootloaderInstallContext(InstallContext):
cmd = 'install' + '_' + x
variant = x
|