****** Building Unix Squeak (>= 3.2) from source ****** Ian Piumarta Last edited: 2009-08-12 11:17:55 by piumarta on emilia.local Translated to .ps/.pdf/.html/.txt: August 12, 2009 ***** Contents ***** * 0_The_easy_way * 1_The_hard_way:_configure,_build,_install * 2_Generating_your_own_VM_and_plugin_sources o 2.1_How_configure_finds_the_src_directory * 3_Adding_your_own_plugins o 3.1_Plugin-specific_configuration o 3.2_Plugin-specific_Makefile_declarations_and_rules o 3.3_Examples_taken_from_existing_plugins o 3.4_Coping_with_VMMaker_quirks o 3.5_If_all_else_fails * Index ****** 0 The easy way ****** Beginning with version 3.7-7, everything you need is included with the Unix sources regardless of how you obtained them - either as a tarball or by repository checkout. (This was done to eliminate frequent problems encountered by people combining the repository Unix and Cross sources with a set of incompatible generated sources.) If you extract a tarball then you will have a top-level directory named Squeak- X.Y-Z (for some values of X, Y, and Z). If you are checking out from a repository, you can call the direcory anything you like; for example: $ svn co http://squeakvm.org/svn/squeak/trunk squeak will leave you with a directory called squeak. (We'll assume from now on that the directory is called squeak.) Next change to the 'unix' directory within the sources that you just checked out (or extracted from the tarball): $ cd squeak/platforms/unix Build the VM and plugins by running make: $ make Then install the VM, plugins and manual pages by running make again (with superuser privileges): $ sudo make install To delete the temporary files created during the build process, run make one last time: $ make clean That's all there is to it. ****** 1 The hard way: configure, build, install ****** Unix Squeak is built using the (almost) universal ``configure; make; make install''. If you haven't come across this before, read on... Create a build directory (which we will call `blddir' from now on) and then `cd' to it: $ mkdir blddir $ cd blddir A convenient place is just next to the platforms directory, like this: $ cd squeak $ ls platforms src ... $ mkdir bld $ cd bld Create the build environment by running the script configure which lives in the platforms/unix/config directory. Note: The configure script accepts lots of options. To see a list of them, run: `configure --help' Assuming you've created the blddir next to platforms, this would be: $ ../platforms/unix/config/configure Note: This assumes that the VMMaker sources are in `../src'. However, since the Unix Squeak support code is independent of the image version from which VMMaker generated the interpreter/plugin sources, it is possible that your source distribution comes with more than one src directory (corresponding to more than one image version used to generate the sources). In such cases you will have to tell configure which source version to use, via the `--with-src' option. For example, if there are two source directories called src-3.2gamma-4857 and src-3.3.alpha-4881 then you would use one of the following commands: $ .../configure --with-src=src-3.2gamma-4857 nbsp;CLASS="textit">or $ .../configure --with-src=src-3.3alpha-4881 Build the VM and plugins by running make: $ make Note: If you want to build just the VM (without external plugins) or just the external plugins (without the VM) then you can use: `make squeak' or `make plugins' respectively. Finally install the VM, plugins and manual pages: $ su root $ make install ****** 2 Generating your own VM and plugin sources ****** Generating your own VM/plugin sources might be necessary for various reasons: * you want to change the mix of internal vs. external plugins * you want to remove some plugins from the VM that you will never use * you've pulled in some updates that modify the Interpreter or plugins * you've filed-in (or written) a whole new plugin * etc... Version 3.2 (and later) of Unix Squeak use VMMaker to generate the core interpreter and plugin sources. Start Squeak in the top-level directory (the one containing the platforms directory); for example: $ ls src platforms ... $ squeak MyCoolPlugin.image Open a VMMakerTool and modify the setup to your liking. Note: The VMMaker configuration used to build the distributions of Unix Squeak is available in platforms/unix/config/VMMaker.config. Then click on the relevant ``generate ...'' button. You can now `configure; make; make install' in your blddir (as described above). Note: You only need to run configure once for a given blddir (on the same host). If you modify the choice of plugins (or change whether they're internal/external) then you can update the build environment by running the config.status script in the bldddir, like this: $ squeak MyCoolPlugin.image ... generate new sources ... $ cd blddir $ ./config.status $ make This is much faster than running configure all over again. (In fact, make should detect any changes to the plugin configuration and re-run config.status for you automatically.) Note: `configure' doesn't actually create any files. The last thing it does is run `config.status' to create the configured files in blddir from the corresponding file.ins in the unix/config directory. So in the remainder of this document the phrase `during configuration' means either when running `configure' for the first time or running `config.status' to update an already configured build environment. ***** 2.1 How configure finds the src directory ***** Starting with version 3.7 configure looks in two places for the src directory, in the following order, and uses the first one that it finds: * the top-level directory (the one containing the platforms directory); * the platforms/unix directory. In other words, if you want to use your own generated sources without deleting the built-in generated sources, generating them into a src directory next to platforms (and then re-running configure) will do what you want. ****** 3 Adding your own plugins ****** Note: This section is intended primarily for plugin developers. If your plugin requires no platform-specific tweaks then there's nothing for you to do. configure (and config.status) will provide a default Makefile for it that should work. If your plugin requires only platform-independent tweaks (and/or additional hand-written code) then these go in platforms/Cross/plugins, and there's nothing for you to do (in Unixland). On the other hand, if you require special configure tests or additional declarations/rules in your plugin's Makefile then you need to specify them explicitly. Note: Unix Squeak subscribes to the following philopsophy: Absolutely everything that is specific to Unix (sources, headers, configure and Makefile extensions, etc.) lives under platforms/unix. In other words: there is not (nor aught there be) any Unix-related information under the platforms/Cross directory. (Unix Squeak is entirely encapsulated under platforms/unix and is utterly immune to ``random junk'' elsewhere in the platforms tree.) First you must create a new directory under platforms/unix/plugins named after your plugin. This directory will hold the files describing the additional configuration checks and/or Makefile contents. For example, if your plugin is called ``MyCoolPlugin'' then $ mkdir platforms/unix/plugins/MyCoolPlugin would be the thing to do. (The following sections will refer to this directory as platdep since the full path is quite a mouthful of typing for my lazy fingers.) ***** 3.1 Plugin-specific configuration ***** Your plugin can ask configure to run additional tests (and to set additional variables in its output files) simply by including a file called acinclude.m4 in it's platdep directory. Note: The configure script is `compiled' from several other files. If you create a `platdep./acinclude.m4' file then you must `recompile' configure. You can do this by `cd'ing to unix/config and running `make', or (if you have GNU make) from the blddir like this: $ make -C ../platforms/unix/config In addition to the usual autoconf macros, the following macros are available specifically for Squeak plugins to use: **** 3.1.1 AC_PLUGIN_CHECK_LIB(lib,func) **** This is similar to the autoconf `AC_CHECK_LIB' macro. func is the name of a function required by the plugin, defined in the external (system) library lib. The macro checks that the library is available (via `- llib') and then adds it to the list of libraries required by the plugin (see the explanation of [plibs] in Section 3.2.1 for a description of how library dependencies for plugins are handled). If func cannot be found in lib then the plugin will be disabled and a message to that effect printed during configuration. (The VM can still be built, without rerunning VMMaker or reconfiguring, and the plugin will simply be ommitted from it.) **** 3.1.2 AC_PLUGIN_DEFINE_UNQUOTED(keyword,text) **** This is similar to the autoconf `AC_DEFINE_UNQUOTED' macro. keyword is a Makefile keyword (usually of the form `[name]') and text is arbitrary text to be associated with it. Calling this macro causes mkmf to substitute text for all occurrences of keyword in the Makefile generated for the plugin. **** 3.1.3 Plugin-specific variables **** The following variables are also set during the execution of a plugin-specific acinclude.m4: ${plugin} is the name of the plugin; ${topdir} is the path to the top-level directory (containing platforms); ${vmmdir} is the path to the VMMaker `src' directory. ***** 3.2 Plugin-specific Makefile declarations and rules ***** Three mechanisms are avilable for this: 1. scanning additional dirrectories for sources and headers; 2. including a few additional lines into the default Makefile; and 3. replacing entirely the default Makefile with a hand-written one. (The last option isn't as scary as it might sound: read on...) **** 3.2.1 The anatomy of a plugin's Makefile **** Before proceeding, let's take a minute to understand how Unix Squeak compiles and links files in its default Makefile for plugins. The default Makefile looks like this: # default Makefile for Unix Squeak plugins [make_cfg] [make_plg] XINCLUDES = [includes] OBJS = [targets] TARGET = [target] PLIBS = [plibs] [make_inc] $(TARGET) : $(OBJS) Makefile $(LINK) $(TARGET) $(OBJS) $(PLIBS) [make_targets] .force : Note: The keywords appearing between `[ square brackets ]' are substituted during configuration by a preprocessor called `mkmf' according to the kind of plugin (internal/external) being built. [make_cfg] is the configured variable section. It contains the platform- specific information gleaned by configure while it was figuring out which compiler you have, what flags your linker needs, where to install stuff, and so on. [make_plg] contains a handful of definitions which depend on whether the plugin is being compiled as internal or external: o the extension for object files a the extension for plugins COMPILE the command to compile a source file into an object file LINK the command to link one or more object files into a plugin For internal plugins: $o is `.o' and $a is `.a'. $(COMPILE) is the C compiler (`$(CC) ... -o', so the first thing after the command must be the output filename) and $(LINK) is archiver (`ar -rc', again requiring the output file to follow immediately). Note that internal plugins are built as `ar' archives before being linked into the final binary. For external plugins: $o is `.lo', $a is `.la', and $(COMPILE) and $(LINK) are invocations of `libtool' to create position-independent objects and shared libraries (with a `-o' appearing right at the end, so the first thing after the command must be the output filename). [includes] is a list of `-Idir' compiler flags, one for each of the directories src/plugins/nbsp;CLASS="textit">name src/vm/intplugins/ nbsp;CLASS="textit">name platforms/Cross/plugins/ nbsp;CLASS="textit">name platforms/unix/plugins/ nbsp;CLASS="textit">name in which at least one header file is present. [targets] is a list of object files corresponding to the source (.c) files found in the directories: src/plugins/nbsp;CLASS="textit">name/*.c src/vm/intplugins/ nbsp;CLASS="textit">name/*.c platforms/Cross/plugins/ nbsp;CLASS="textit">name/*.c platforms/unix/plugins/ nbsp;CLASS="textit">name/*.c where each source file has been stripped of the directory name and had the `.c' converted into `$o'. [target] is the name of the plugin, including the $a extension. [plibs] is a list of zero or more libraries on which the plugin depends (as detected using the macro AC_PLUGIN_CHECK_LIB in the plugin-specific acinclude.m4). If the plugin is being built internally then this list is empty and the required libraries are included in the final link command. If the plugin is being built externally then the plugin itself (a shared object) is linked against these libraries (via [plist]) rather than with the main VM binary. (This is to ensure that a missing shared object needed by an external plugin will only affect the operation of that plugin and not prevent the rest of the VM from running, which would be the case if the entire VM were linked against it.) [make_inc] is the contents of the Makefile.inc file in your plugin's platdep directory (or empty if this file doesn't exist). [make_targets] is a list of rules for building the files listed in [targets]. Each rule looks like this: name$o : nbsp;CLASS="textit">original/source/dir/name.c $(COMPILE) name$o nbsp;CLASS="textit">original/source/dir/name.c **** 3.2.2 A note about $(COMPILE) and $(LINK) commands **** You should never pass additional flags to these commands explicitly. This is because you cannot know how they are defined. (Their definitions depend on whether the plugin is being built internally or externally -- and might even change radically in future releases of Unix Squeak.) Instead you should pass additional compiler/linker flags to these commands by setting the following variables in `Makefile.inc' or `Makefile.in': XCPPFLAGS `-I' flags for cpp XDEFS `-D' flags for cpp XCFLAGS anything to be passed to the compiler XLDFLAGS anything to be passed to the linker Note: `mkmf' already uses `XINCLUDES' to pass the list of directories containing plugin header files to cpp. You can redefine it if you like, but make sure that `[includes]' appears in its definition (or in the definition of `XCPPFLAGS'). **** 3.2.3 Specifying additional source directories **** mkmf looks for a file in your plugin's platdep directory called `mkmf.subdirs'. If this file exists then it should contain a list of directory names relative to the top-level directory (the one containing the src and platform directories). These directories will be added to the list of locations searched for `.c' and `.h' files while constructing the substitutions for `[includes]', `[targets]' and `[make_targets]'. **** 3.2.4 Including additional material in the default Makefile **** If the file platdep/Makefile.inc exists then mkmf will substitute its contents into the Makefile in place of the [make_inc] keyword. Note: Makefile.inc is read into the Makefile under construction before mkmf performs substitutions on the `[keyword]'s. In other words, your Makefile.inc can use the above keywords to include relevant declarations and rules without worrying about whether the plugin is internal or external. **** 3.2.5 Replacing the default Makefile entirely **** If neither of the above are sufficient then you can create a complete Makefile template called platdep/Makefile.in. mkmf will use this template instead of the default Makefile template shown earlier, and will perform keyword substitutions on it as described above to create the final Makefile. (In other words, simply copying the default template shown earlier will result in a Makefile identical to the one that mkmf would have produced by default. ***** 3.3 Examples taken from existing plugins ***** By way of example we'll look at how two existing plugins specialise their configuration and Makefiles. **** 3.3.1 Configuration **** The B3DAcceleratorPlugin requires OpenGL in order to compile. The file unix/ plugins/B3DAcceleratorPlugin/acinclude.m4 contains a single call to an autoconf-style macro: AC_PLUGIN_SEARCH_LIBS(glIsEnabled, GL) This works similarly to the autoconf `AC_SEARCH_LIBS' macro: If a library libGL.{a,so} (OpenGL) exists and exports the function glIsEnabled() then `-lGL' is added to the final VM link command. Otherwise the plugin is disabled (and a message warning of the fact is printed). Note: There's a bug here. This should also check for `GL_VERSION_1_1' in headers. **** 3.3.2 Customising the Makefile **** The Mpeg3Plugin requires a (modified) libmpeg to be compiled along with it. The sources for this library are in (several) subdirectories of Cross/Meg3Plugin and they require additional cpp definitions in order to compile correctly. To cope with the additional directories, unix/plugins/Mpeg3Plugin/mkmf.subdirs simply lists them: platforms/Cross/plugins/Mpeg3Plugin/libmpeg platforms/Cross/plugins/Mpeg3Plugin/libmpeg/audio platforms/Cross/plugins/Mpeg3Plugin/libmpeg/video To cope with the additional cpp definitions, we could have written a tiny Makefile.inc containing: XDEFS = -DNOPTHREADS Unfortunately the additional source directories contain various utility and test programs (which must not be built) so we cannot rely on mkmf generating the correct [targets] list. Instead we just copy the default Makefile ``template'' (shown above) as Mpeg3Plugin/Makefile.in and insert the required list of targets (and cpp definition) manually. The end result looks like this: # Makefile.in for Mpeg3Plugin in Unix Squeak [make_cfg] [make_plg] TARGET = Mpeg3Plugin$a PLUGIN = Mpeg3Plugin$o VIDEO = getpicture$o headers$o idct$o macroblocks$o nbsp;CLASS="textit">etc... AUDIO = dct$o header$o layer1$o layer2$o layer3$o nbsp;CLASS="textit">etc... LIBMPEG = bitstream$o changesForSqueak$o libmpeg3$o nbsp;CLASS="textit">etc... OBJS = $(PLUGIN) $(VIDEO) $(AUDIO) $(LIBMPEG) XINCLUDES = [includes] XDEFS = -DNOPTHREADS $(TARGET) : $(OBJS) Makefile $(LINK) $(TARGET) $(OBJS) [make_targets] .force : Note: The default `[make_targets]' will contain additional rules for the objects that we're trying to avoid building (because it's built from an exhaustive list of `.c' files in the source directories). This does no harm since the offending rules can never be triggered (their targets are not listed in `OBJS'). ***** 3.4 Coping with VMMaker quirks ***** VMMaker will refuse to compile a plugin if it thinks the plugin requires platform support. This is ``all-or-nothing'': if platform support is required on one platform then it is required on all platforms (even if the plugin compiles quite happily without platform support in Unix). The easiest way to add ``null'' platform support is to place an empty `Makefile.inc' in the plugin's platdep directory. (To see this in action, look in unix/plugins/JPEGReadWriter2Plugin.) ***** 3.5 If all else fails ***** (Where ``all else failing'' is defined as: ``after trying for 20 minutes and still getting nowhere''.) If you're writing a plugin that needs platform support (beyond dumb inclusion of a few additional `.c' files) and this document has been of no help at all (or if you understood it but you're still suffering from ``all else failing'') then send me mail and I'll be happy to help you with the various platdep files. ***** Index ***** $(COMPILE) 3.2.2 $(LINK) 3.2.2 $(XCFLAGS) 3.2.2 $(XCPPFLAGS) 3.2.2 $(XDEFS) 3.2.2 $(XINCLUDES) 3.2.2 $(XLDFLAGS) 3.2.2 [includes] 3.2.1 [make_cfg] 3.2.1 [make_inc] 3.2.1 [make_plg] 3.2.1 [make_targets] 3.2.1 [plibs] 3.2.1 [target] 3.2.1 [targets] 3.2.1 AC_PLUGIN_CHECK_LIB 3.1.1 AC_PLUGIN_DEFINE_UNQUOTED 3.1.2 acinclude.m4 3.1 example 3.3.1 additional plugin source directories 3.2.3 build directory configuring 1 creating 1 config.status 2 versus configure 2 configure 3.1 macros for plugins 3.1 recreating 3.1 emergency services 3.5 Makefile 3.2.1 avoiding $(XINCLUDES) 3.2.2 compile/link commands 3.2.2 keyword substitution 3.2.1 passing extra flags 3.2.2 replacing 3.2.5 target rules 3.2.1 Makefile keywords [includes] 3.2.1 [make_cfg] 3.2.1 [make_inc] 3.2.1 [make_plg] 3.2.1 [make_targets] 3.2.1 [plibs] 3.2.1 [target] 3.2.1 [targets] 3.2.1 Makefile.in 3.2.5 example 3.3.2 Makefile.inc 3.2.4 example 3.3.2 keyword substitution 3.2.4 mkmf 3.2.1 additional source directories 3.2.3 default header directories 3.2.1 default source directories 3.2.1 mkmf.subdirs 3.2.3 example 3.3.2 plugin Makefile 3.2 Makefile anatomy 3.2.1 adding your own 3 configuring 3.1 target rules 3.2.1 Unix-specific directory 3 Unix-specific files 3 VMMaker configuration file 2 missing platform support 3.4 reference 2 =============================================================================== 2009-08-12