1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
|
.. index::
single: script (tutorial)
single: ECF_HOME (tutorial)
.. _tutorial-defining-a-task:
Defining the first task
=======================
With the :term:`suite definition` ready, the next step is to create the :term:`task` script, also called
:term:`ecf script`. This script typically uses extension :code:`.ecf`.
Task Script Structure
---------------------
A common pattern used for defining an :term:`ecf script` is to consider three sections:
1. A *head* section, that sets up the necessary execution environment, including error handling,
for the :term:`task`; and eventually notifies the :term:`ecflow_server` that the :term:`task` has started
2. A *body* section, which contains the actual work to be done by the :term:`task`
3. A *tail* section, that performs the necessary clean up for the :term:`task`; and eventually notifies
the :term:`ecflow_server` that the :term:`task` has completed
The *head* and *tail* sections are common amongst all :term:`tasks <task>`, while the *body* section is specific to each :term:`task`.
To avoid code duplication, ecFlow enables placing the *head* and *tail* sections in separate files, and then include them in the :term:`ecf script` using "C-like" preprocessing :term:`directives`.
The process of preprocessing the :term:`ecf script` and including *head* and *tail* files is called the :term:`job file <job file>` generation.
Task Head and Tail
------------------
The *head* and *tail* sections are placed in separate files, called include files.
These files use :term:`variables <variable>` as a configuration mechanism. The :term:`variables <variable>`
are introduced in between %% characters (e.g. :code:`%VARIABLE_NAME%`) and are substituted when the :term:`job file <job file>`
is created.
These files are introduced into the :term:`ecf script` using the :code:`%include` directive.
.. note::
The use of :code:`%include` directives is a generic code reuse mechanism, and can be used to avoid code duplication.
Whenever several :term:`tasks <task>` need to perform the same operation, this can be placed in a separate file,
and then included in the :term:`ecf script` using the :code:`%include` directive.
.. tabs::
.. tab:: head.h
The following :file:`head.h` file, containing the *header* section, performs the following at the start of the :term:`ecf script`:
* Setup the environment for communication with the :term:`ecflow_server`
* Define script error handling, e.g. when the script fails a trap is raised, and the server is informed that the :term:`task` has :term:`aborted`.
* Inform the server that job has started, using the :code:`init` :term:`child command` .
.. note::
Notice how variables, such as :code:`ECF_HOST` and :code:`ECF_PORT`, are used in the script.
These variables are automatically generated by ecFlow and can be used as part of any section of the :term:`ecf script`.
.. code-block:: shell
:linenos:
:caption: $HOME/course/head.h
#!/usr/bin/env bash
set -e # stop the shell on first error
set -u # fail when using an undefined variable
set -x # echo script lines as they are executed
#
# Important: Adjust the following to point at the ecFlow install location
#
ECFLOW_ROOT_DIR=/path/to/ecflow/%ECF_VERSION%
# Define the variables that are needed for any communication with ECF
export ECF_PORT=%ECF_PORT% # The server port number
export ECF_HOST=%ECF_HOST% # The name of ecf host that issued this task
export ECF_NAME=%ECF_NAME% # The name of this current task
export ECF_PASS=%ECF_PASS% # A unique password
export ECF_TRYNO=%ECF_TRYNO% # Current try number of the task
export ECF_RID=$$
# Define the path where to find ecflow_client
# make sure client and server use the *same* version.
# Important when there are multiple versions of ecFlow
export PATH=${ECFLOW_ROOT_DIR}/bin:$PATH
# Tell ecFlow the task has started
ecflow_client --init=$$
# Define a error handler
ERROR() {
set +e # Clear -e flag, so we don't fail
wait # wait for background process to stop
ecflow_client --abort=trap # Notify ecFlow that something went wrong
trap 0 # Remove the trap
exit 0 # End the script
}
# Trap any calls to exit and errors caught by the -e flag
trap ERROR 0
# Trap any signal that may cause the script to fail
trap '{ echo "Killed by a signal"; ERROR ; }' 1 2 3 4 5 6 7 8 10 12 13 15
.. tab:: tail.h
The following :code:`tail.h` file, containing the *tail* section, performs the following at the end of :term:`ecf script`:
* Waits for any background processes to complete
* Informs the server that the :term:`task` has completed successfully
* Cleans up any error handling previously defined
.. code-block:: shell
:linenos:
:caption: $HOME/course/tail.h
wait # wait for background process to stop
ecflow_client --complete # Notify ecFlow of a normal end
trap 0 # Remove all traps
exit 0 # End the shell
Task Script
-----------
The following :file:`t1.ecf` file is the :term:`ecf script` for the :term:`task` :code:`t1`.
.. tabs::
.. tab:: t1.ecf
This script includes the *head* and *tail* sections from the files :file:`head.h` and :file:`tail.h`, respectively.
The *body* section of this script simply prints a message to standard output, including the value of the variable :code:`ECF_HOME`.
ecFlow expects task files to be in a directory structure under :code:`ECF_HOME` that reflects the hierarchy in the suite.
In the example, task :code:`t1` is part of the suite :code:`test`, so the script for task :code:`t1` is expected to
be in sub-directory :code:`test`.
.. code-block:: shell
:linenos:
:caption: $HOME/course/test/t1.ecf
%include "../head.h"
echo "I am part of a suite that lives in %ECF_HOME%"
%include "../tail.h"
.. important::
When using the :code:`%include "<filename>"` directive, the path to the include file is given relative to
the location of the :term:`ecf script` file, thus the :code:`../` in the example above.
.. note::
Notice how the :code:`ECF_HOME` variable is used in the script. This variable is defined in the :term:`suite definition` file.
**What to do:**
* Create the :code:`$HOME/course/head.h` and :code:`$HOME/course/tail.h` include files based on the examples provided above.
Make sure to adjust the :code:`ECFLOW_ROOT_DIR` variable in the :file:`head.h` file to point to the correct ecFlow installation location.
* Create the directory structure, and the :code:`$HOME/course/test/t1.ecf` task script file based on the example provided above.
|
