1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
===============
Getting started
===============
.. toctree::
:hidden:
INSTALL
notebooks
Core concepts (the philosophical bit)
=====================================
On a fundamental level, converting between different programming
styles in a low-level compiled language like Fortran or C/C++ typically
requires assumptions to be made that are specific to the data and algorithm
and do not generalize to the entire language. This is why Loki provides a
programmable interface rather than a push-button solution, leaving it
up to developers to decide which assumptions about the original source
code can be used and how.
For example, converting large numbers of IFS physics code to a "single column"
format (see below) requires the explicit knowledge of which index variables
typically represent the parallel dependency-free horizontal dimension that
is to be lifted.
The aim of Loki is therefore to give developers all the tools to encode their
own code transformation in an elegant, pythonic fashion. The core concepts
provided for this are:
* :any:`Module` and :any:`Subroutine` classes (kernels) that each provide an
:doc:`Intermediate Representation (IR) <internal_representation>`
of their source code, as well as
utilities to inspect and transform the underlying IR nodes.
* Expressions contained in IR nodes, such as :any:`Statement`, :any:`Loop`,
and :any:`Conditional`, are represented as independent sub-trees, based on the
`Pymbolic <https://github.com/inducer/pymbolic>`__ infrastructure.
* Three frontends are supported that are used to parse Fortran code
either from source files or strings into the Loki IR trees. Multiple
backends are provided to generate Fortran or (experimentally) C or (even more
experimentally) Python code from the combined IR and expression trees.
* A :any:`Transformation` class is provided that allows users to encode
individual code changes based on the abstract representation
provided by Loki's IR and expression objects and can be applied
to individual :any:`Subroutine` and :any:`Module` objects - much like simple
compiler passes.
* A :any:`Scheduler` class that provides bulk processing
and inter-procedural analysis (IPA) tools to apply individual changes
over large numbers of files while honoring the call-tree that
connects them.
Example transformations and current features
============================================
Loki is primarily an API and toolbox, allowing developers to create their
own head scripts and to create and invoke source-to-source translation toolchains.
In addition, a set of supported transformations is provided by the
package itself in :mod:`loki.transformations`. These range from utilities
that can be used with generic Fortran codes to highly bespoke transformations
for generating GPU code based on highly model-specific assumptions.
The ``loki_transform.py`` script is provided by the Loki install. The primary
transformation passes provided by these example transformations are:
* **Idempotence (Idem)** - A simple transformation that performs a
neutral parse-unparse cycle on a kernel.
* **Single column abstraction (SCA)** - Transforms a set of kernels
into Single column format by removing the specified horizontal
iteration dimension. This transformation has a "driver" and a
"kernel" mode, as it potentially changes the subroutine's call
signature to remove derived types (structs do not expose
dimensions).
* **Single column coalesced (SCC)** - Transforms a set of kernels
from CPU-style (SIMD) vectorization format to a GPU-style (SIMT)
loop layout. It removes the specified horizontal iteration
dimension and re-inserts it outermost. Optionally, the horizontal
loop can be stripped from kernels and re-inserted in the driver, to
allow hoisting the allocation of temporary arrays to driver level (SCCH).
* **C transpilation** - A dedicated Fortran-to-C transpilation
pipeline that converts Fortran source code into (column major,
1-indexed) C kernel code. The transformation pipeline also creates
the necessary header and `ISOC` wrappers to integrate this C kernel
with a Fortran driver layer, as demonstrated with the
`CLOUDSC ESCAPE dwarf <https://github.com/ecmwf-ifs/dwarf-p-cloudsc>`_.
First steps
===========
To start using Loki, follow the :doc:`installation instructions <INSTALL>`.
We recommend to study the :doc:`Jupyter notebooks <notebooks>` in the `example`
directory to get familiar with the basic API of Loki. The
:doc:`Using Loki <using_loki>` section provides more details on the inner
workings and underpinning concepts.
Contributions
=============
Contributions to Loki are welcome. In order to do so, please open an
issue in the `Github repository <https://github.com/ecmwf-ifs/loki/issues>`__
where a feature request or bug can be discussed.
Then create a pull request with your contribution. We require you to read and sign the
`contributors license agreement (CLA) <http://claassistant.ecmwf.int/ecmwf-ifs/loki>`__
before your contribution can be reviewed and merged.
|
