<!--  vim: set sw=2 sts=2 et ft=docbk:

  Part of the A-A-P recipe executive: Adding a Language

  Copyright (C) 2002-2003 Stichting NLnet Labs
  Permission to copy and use this file is specified in the file COPYING.
  If this file is missing you can find it here: http://www.a-a-p.org/COPYING

-->

<para>
  This chapter explains how to add support for a language to &Aap;.
  Both for your own setup and for a module that is to be included in an &Aap;
  release.
</para>
<para>
  Although the text explains modules for a language, this can also be used for
  adding a module for other purposes.  It is a generic method to have &Aap;
  load a module that defines actions, sets variables, etc.
</para>
<para>
  In a recipe a module can be used with the
  <link linkend="cmd-import">:import</link> command.
  Thus when your recipe uses Java, you would add this line:
</para>
<programlisting>
      :import java
</programlisting>

<para>
  For the standard modules see <xref linkend="ref-modules"/>.
</para>


<bridgehead>Module Name</bridgehead>

<para>
  It is important to chose a good name for the module, because it must be
  unique on your system.  For a programming language simply use the name of
  the language.  For Java you would use "java", for the D language "d".  In
  case there are variants of a language that are different enough to justify
  using a different module, add something to the name to make this clear.
  You can use letters, digits and the underscore.  Examples: "foo", "foo2004",
  "foo_latest".
</para>
<para>
  If you have a specific module with different functionality, you could
  prepend "my".  For example, "myjava".
</para>
<para>
  After the module has been imported, the variables in it can be accessed
  through the "m_name" scope, where "name" is the name of the module.  For
  example the variable "$m_d.DMD" holds the command used for the "d" language
  to execute the compiler.
</para>
<para>
  Note: It is not possible to have a variable and a scope with the same name.
  Thus if you have a module called "foo", which uses the scope "m_foo", you
  cannot have a variable "m_foo" anywhere in a recipe.
</para>
<para>
  Since the modules are &Aap; recipes the file names always end in ".aap".
</para>


<bridgehead>Module directories</bridgehead>

<para>
  Modules that are included with &Aap; are located in the &Aap; sub-directory
  "modules".  If you are working on a module that is to be submitted for a
  next &Aap; release you may also want to put the module there for testing.
  But be aware that this directory may be overwritten or deleted when updating
  &Aap;!
</para>

<para>
  Modules that you want to share with all users on your system should be
  placed in a system-wide directory.  For Unix this usually is
  <filename>/usr/local/share/aap/modules</filename>.  This can be changed with
  <link linkend="var-aapsysdir">$AAPSYSDIR</link>.
</para>

<para>
  Modules that you only use for yourself are placed in
  <filename>$HOME/.aap/modules</filename> (for Unix and Mac OS X),
  <filename>$HOME/aap/modules/</filename>,
  <filename>$HOMEDRIVE/$HOMEPATH/aap/modules/</filename> or
  <filename>c:/aap/modules/</filename> (MS-Windows).
</para>

<para>
  If you only want to change a few settings after the module is loaded, you
  can place these in the <filename>modules2</filename> directory. See
  <link linkend="cmd-import">:import</link> for details.
</para>


<bridgehead>Parts Of A Module</bridgehead>

<para>
  In a module you will usually find these parts:
<variablelist>
  <varlistentry>
    <term>(1) filetype recognition</term>
    <listitem><para>How to recognize files used in the module.  For a module
        included with &Aap; this is normally missing, since it can be added to
        the generic Python filetype module.
        See <xref linkend="user-filetype"/>.
    </para></listitem>
  </varlistentry>

  <varlistentry>
    <term>(2) default values</term>
    <listitem>
      <para>
        Give default values to variables used in the commands further on.  The
        user can change these values if he needs to.  It is a good habit to
        put all specific command names, directories and arguments in a
        variable.
      </para>
    </listitem>
  </varlistentry>

  <varlistentry>
    <term>(3) object file suffixes</term>
    <listitem>
      <para>
        If the language uses different suffixes for object files, for static
        or dynamic libraries, this can be specified.
        See <link linkend="lang-suffix">below</link>.
      </para>
    </listitem>
  </varlistentry>

  <varlistentry>
    <term>(4) search for tools</term>
    <listitem>
      <para>
        Usually there are different ways to invoke the compiler on different
        platforms.  And on one platforms various tools may be installed.  The
        <link linkend="cmd-toolsearch">:toolsearch</link> command is used to
        locate the tools.
        See <link linkend="lang-toolsearch">below</link>.
      </para>
    </listitem>
  </varlistentry>

  <varlistentry>
    <term>(5) actions, rules and routes</term>
    <listitem>
      <para>
        The generic methods for compiling and building.
        See <link linkend="lang-actions">below</link>.
      </para>
    </listitem>
  </varlistentry>

</variablelist>
</para>


<bridgehead>Example</bridgehead>

<para>
  This example is for the "D" language.  This module is included with &Aap;.
</para>

<programlisting>
      # (1) Filetype recognition
      :filetype
          suffix d d

      # (2) Default values
      DMD = dmd

      # (3) Object file suffixes
      # dll and lib objects are equal to normal objects.
      _top.D_LIBOBJSUF = $OBJSUF
      _top.D_DLLOBJSUF = $OBJSUF

      # (4) Search for tools
      # Actions are installed for every toolchain that exists.
      # The first one found sets $D_COMPILE_ACTION and friends.
      :toolsearch dmd

      # (5) Actions, Rules and Routes

      # :do compile

      :action compile object,libobject,dllobject,default d
          @if not _no.get("target"):
              target = `src2obj(fname)`
          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target
          @if _no.get("D_COMPILE_ACTION"):
              :do $D_COMPILE_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DFLAGS -of$target -c $source

      :rule {global} {default} %$OBJSUF : {buildcheck = $DMD $?DFLAGS } %.d
          :do compile {target = $target} $source

      # :do build for object files resulting from "d" source files.

      :action d_build default
          @if _no.get("D_BUILD_ACTION"):
              :do $D_BUILD_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DLINKFLAGS -of$target $source

      # default route

      :route {default} d object,libobject,dllobject
           compile
</programlisting>


<bridgehead id="lang-suffix">Object File Suffixes</bridgehead>

<para>
  Many languages use the same object file suffixes as used for C: ".o" on Unix
  and ".obj" on MS-Windows.  But for some languages a different suffix is
  used and the suffixes for building a static and shared library are
  different.
</para>

<para>
  The simplest example is when a language always uses the same suffix for all
  kinds of object files.  For example, the "d" language does not make a
  difference between normal object files, static library objects, etc.  This
  can be specified like this:
</para>

<programlisting>
     _top.D_DLLOBJSUF = $OBJSUF
     _top.D_LIBOBJSUF = $OBJSUF
</programlisting>

<para>
  Note that the "_top" scope is used.  If this would be left out the variables
  would only be set in the scope of the module and would not be available in
  the user recipe.
</para>

<para>
  The name of the variable is made from the filetype in upper case, and
  underscore and the name of the generic variable.
  The <link linkend="python-src2obj">src2obj()</link> function will look for
  these variables.
  Note that the <emphasis>filetype</emphasis> is used, not the module name!
</para>

<para>
  An extreme example is when all suffixes are different:
</para>

<programlisting>
     _top.FOO_OBJSUF = .fo
     _top.FOO_DLLOBJSUF = .dfo
     _top.FOO_LIBOBJSUF = .lfo
     _top.FOO_LTOBJSUF = .tfo
</programlisting>

<para>
  An advantage of using a different suffix for every type of object file is
  that all types can be generated in the same directory.
</para>

<para>
  If the compiler or linker does not support using different suffixes, but the
  various object files are different, the user must make sure that the object
  files are put in different build directories.  This can be done by
  specifying an attribute on the source files to use a different build
  directory:
</para>

<programlisting>
    :program foo : foo.c                    # uses $BDIR
    :dll foo : foo.c {var_BDIR = $BDIR-dll}
    :lib foo : foo.c {var_BDIR = $BDIR-lib}
</programlisting>


<bridgehead id="lang-toolsearch">Searching For Tools</bridgehead>

<para>
  When there is only one compiler that always uses the same (type of)
  arguments you can simply invoke it directly from the actions in the module.
  Otherwise, &Aap; can search for tools that are currently installed.
  The working of tools is explained in <xref linkend="user-tools"/>
</para>

<para>
  As an example, let's look at how &Aap; searches for C compilers:
</para>

<programlisting>
      @if osname() == 'mswin':
          :toolsearch msvc mingw gcc icc bcc
      @elif osname() == 'os2':
          :toolsearch icc gcc msvc
</programlisting>

<para>
  The <link linkend="cmd-toolsearch">:toolsearch</link> command is used with a
  list of tools that need to be checked for.  The first tool for which the
  exists() function returns True is then used.  This works by setting
  variables to the name of the action to be used.  For example, the "msvc"
  tool sets $C_COMPILE_ACTION to "compile_msvc".
</para>
<para>
  The list of tools to be searched depends on the platform.  Note that the
  "msvc" tool is the first choice for MS-Windows, but the last choice for
  OS/2.  This ordering is based on what works best for most people.
</para>
<para>
  Note that for Unix systems there is no search for tools.  That is because
  the default command for the C compiler "always" works there.
</para>
<para>
  There is one trick you should be aware of, which is using the "buildaction"
  attribute on object files.  When a specific compiler is used to turn a
  source file into an object file, this often means a specific action needs to
  be used to turn this object file into a program or library.  The
  compile action must then add a "buildaction" attribute to the object file,
  so that the generic build action can inspect the object files and invoke the
  proper build action.  You can see this in the example above: The compile
  action for the "d" language contains:
</para>
<programlisting>
          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target
</programlisting>
<para>
  The action is not executed when the target is already up-to-date.  Therefore
  the "buildaction" attribute should also be set by the dependency or rule,
  using a ">always" section.  See <xref linkend="build-sections"/>.
  Example:
</para>
<programlisting>
          :rule %$OBJSUF : $.cpp
              >always
                  :attr {buildaction = cxx_build} $target
              >build
                  :do compile {target = $target} $source
</programlisting>
<para>
  Note that the generic build action cannot handle the situation that object
  files with different "buildaction" attributes are linked together.  And
  object files that do not have a "buildaction" attribute are taken along,
  &Aap; assumes they can be linked together with the object files that specify
  the "buildaction" attribute.  If this does not do what you intend, you can
  specify another build action to use with $BUILD_ACTION.  Similarly,
  $BUILDDLL_ACTION and $BUILDLIB_ACTION are used for building libraries.  Look
  in the distributed <filename>default.aap</filename> for the details.
</para>


<bridgehead id="lang-actions">Actions, Rules and Routes</bridgehead>

<para>
  Actions are used to specify how files are to be compiled and build into a
  program or library.  Rules are used to specify which actions to use for
  files matching a pattern.  Routes are used to specify which actions to use
  to turn one filetype into another.  Thus these three items are closely
  related.
</para>

<para>
  The most obvious thing for a language module is to define an action for
  compiling the language.  Here is an excerpt from the "d" module:
</para>

<programlisting>
      :action compile object,libobject,dllobject,default d
          @if not _no.get("target"):
              target = `src2obj(fname)`
          # Use the d_build action for building and also for :dll and :lib.
          :attr {buildaction = d_build} $target
          @if _no.get("D_COMPILE_ACTION"):
              :do $D_COMPILE_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DFLAGS -of$target -c $source
</programlisting>

<para>
  This action is specified for all kind of object files, including dllobject
  and libobject.  If compiling for a library has to be done differently you
  should specify a separate action for libobject and/or dllobject.
</para>

<para>
  The action starts with checking if $target is set.  In rare cases it isn't
  set and needs to be derived from $source.  Since the object file suffixes
  have been defined (see <link linkend="lang-suffix">above</link>) calling
  <link linkend="python-src2obj">src2obj()</link> takes care of this.
  $fname is the first item in $source.
</para>

<para>
  When a D file has been compiled into an object file it needs to be build in
  a special way.  To accomplish this the "buildaction" attribute of the target
  is set to "d_build".  This action is also defined in the D module.
</para>

<para>
  The check for $D_COMPILE_ACTION takes care of invoking the action that was
  defined by a tool.  The first tool found with
  <link linkend="cmd-toolsearch">:toolsearch</link> will set this variable.
  It can later be changed with a
  <link linkend="cmd-usetool">:usetool</link> command.
</para>

<para>
  When $D_COMPILE_ACTION is not set, the default command to compile a D file
  is executed with <link linkend="cmd-sys">:sys</link>.  This happens when no
  appropriate tool could be found.  Often this is used for Unix, where the
  compile command is the same for various compilers.  The user may set
  $m_d.DMD to the name of the compiler to be used and set $DFLAGS
  to additional arguments.  What could be added here is using $DINCLUDE and
  $DDEFINE, like what <link linkend="var-include">$INCLUDE</link> and
  <link linkend="var-define">$DEFINE</link> do for C.
  A compiler may also use <link linkend="var-debug">$DEBUG</link> and
  <link linkend="var-optimize">$OPTIMIZE</link>.  Note that these can't be
  directly used as an argument but must be translated.
</para>

<para>
  The other actions defined in the module are similar.  You can see in the
  example for the D module how the build action is defined:
</para>

<programlisting>
      :action d_build default
          @if _no.get("D_BUILD_ACTION"):
              :do $D_BUILD_ACTION {target = $target} $source
          @else:
              :sys $DMD $?DLINKFLAGS -of$target $source
</programlisting>

<para>
  Defining a rule is optional.  The
  <link linkend="cmd-rule">:rule</link> command selects the source and target
  based on a match of the file name with a pattern.  This is useful when
  a list of source files is defined without telling how they are to be turned
  into object files.  This is the way the rule is defined in the D module:
</para>

<programlisting>
      :rule {global} {default} %$OBJSUF : {buildcheck = $DMD $?DFLAGS } %.d
          :do compile {target = $target} $source
</programlisting>

<para>
  The {global} option is needed to make the rule available everywhere
  (otherwise it would only work in the module recipe).  The {default} option
  allows overruling this rule without getting a warning.
</para>

<para>
  The <link linkend="cmd-route">:route</link> command is used to tell &Aap;
  how to turn a source file into an object file.  This is used for the
  <link linkend="cmd-program">:program</link>,
  <link linkend="cmd-dll">:dll</link> and
  <link linkend="cmd-lib">:lib</link> commands.  For the D language it's
  simply a matter of invoking the compile action.  for other languages it may
  consist of several steps with intermediate results.
</para>

<programlisting>
      :route {default} d object,libobject,dllobject
           compile
</programlisting>