1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204
|
\newpage
%
\chapter{Compiling, running and debugging}
\label{sec_compilationrunning}
\section{Compiling OASIS3-MCT}
\label{subsec_compile}
Compiling OASIS3-MCT libraries can be done in directory {\tt
oasis3-MCT/util/make\_dir} with Makefile \\ {\tt TopMakefileOasis3}
which must be completed with a header file {\tt make.{\it
your\_platform}} specific to the compiling platform used and
specified in {\tt oasis3-MCT/util/make\_dir/make.inc}. One of the
header files distributed with the release can by used as a template.
The root of the OASIS3-MCT tree can be anywhere and must be set in the
variable {\tt COUPLE} in the {\tt make.{\it your\_platform}} file.
The following commands are available:
\begin{itemize}
\item {\tt make -f TopMakefileOasis3} or {\tt make oasis3\_psmile -f
TopMakefileOasis3} (for backward compatibility):
compiles all OASIS3-MCT libraries {\it mct}, {\it scrip} and {\it
psmile};
\item {\tt make realclean -f TopMakefileOasis3}:
removes all OASIS3-MCT compiled sources and librairies.
\end{itemize}
Log and error messages from compilation are saved in {\tt /util/make\_dir} in the files
{\tt COMP.log} and {\tt COMP.err} .
During compilation, a new compiling directory, defined by variable
{\tt ARCHDIR} is created. After successful compilation, resulting
libraries are found in the compiling directory in {\tt /lib} and
object and module files in {\tt /build}.
If module {\tt mod\_oasis} is used in the
models, it is enough to include the path of the psmile objects and
modules ({\tt \$ARCHDIR/build/lib/psmile.MPI1}) for the compilation and to use the psmile library {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} when
linking. If module {\tt mod\_prism} is used in the models, it is necessary to
include the path of the psmile and of the mct objects and modules for
the compilation (i.e. {\tt \$ARCHDIR/build/lib/psmile.MPI1} and {\tt /mct} and to use both the psmile and mct libraries {\tt \$ARCHDIR/lib/libpsmile.MPI1.a} and {\tt libmct.a} and {\tt libmpeu.a} when linking.
Exchange of coupling fields in single and double precision is now supported directly through the interface
(see section \ref{subsubsec_Declaration}), even if all real variables are now treated in double precision internally.
For double precision coupling field, there is no need anymore to promote REAL variables to double precision at compilation.
\section{CPP keys}
\label{subsec_cpp}
The following CPP keys are coded in OASIS3-MCT and can be specified in
{\tt CPPDEF} in {\tt make.{\it your\_platform}} file.
\begin{itemize}
\item {\tt TREAT\_OVERLAY}: To ensure, in {\tt SCRIPR/CONSERV}
remapping (see section \ref{subsec_interp}), that if two cells of
the source grid overlay, at least the one with the greater numerical
index is masked (they also can be both masked); this is mandatory
for this remapping. For example, if the grid line with {\it i=1} overlaps
the grid line with {\it i=imax}, it is the latter that must be masked;
when this is not the case with the mask defined in {\it masks.nc},
this CPP key forces these rules to be respected.
\item {\tt balance}: Add a MPI\_Wtime() function before and after
mct\_isend (MPI put) and mct\_recv (MPI get) to calculate the time
of the send and receive of a coupling field. This option can be used
to produce timestamps in OASIS3-MCT debug files. During a post-processing
phase, this information can be used to perform an analysis of the
coupled components load (un)balance; specific tools have been
developed to do this and will be released with a further version of
OASIS3-MCT. {\bf This option is temporarily not recommended as it was observed that
it was increasing the simulation time of coupled models on
the PRACE computer MareNostrum.}
\end{itemize}
\section{Running OASIS3-MCT}
\label{subsec_running}
Examples of running environement are provided with the sources. For
more details, see the instructions on OASIS web site at
{\tt https://verc.enes.org/oasis/oasis-dedicated-user-support-1/} {\tt user-toys/tutorial-and-test\_interpolation-of-oasis3-mct-1}.
.
\subsection{The tutorial toy model}
\label{subsec_tutorial}
A practical example on how to run OASIS3-MCT and running it in a
coupled system is provided in {\tt oasis3-mct/examples/tutorial}. See also the document {\tt tutorial\_oasis3-mct.pdf} there in for
more details,
\subsection{The test\_interpolation environment}
\label{subsec_testinterpolation}
This environment available with the sources in {\tt
oasis3-mct/examples/test\_interpolation} allows the user to test the
quality of the interpolations and transformations between his source
and target grids by calculating the error of interpolation on the
target grid. For more details, see also the document {\tt test\_interpolation\_oasis3-mct.pdf} there in.
\section{Debugging}
\label{subsec_debug}
\subsection{Debug files}
If you experience problems while running your coupled model with
OASIS3-MCT, you can obtain more information on what is happening by
increasing the {\tt \$NLOGPRT} value in your {\it namcouple} (see also section
\ref{subsec_namcouplefirst}).
Different outputs are written depending on {\tt \$NLOGPRT} value:
\begin{itemize}
\item {0} : one file debug.root.xx is open by the master process of
each model and one file debug\_notroot.xx is open for all the other
processes of each model to write only error information if an error
occurs.
\item {1} : one file debug.root.xx is open by the master process of
each model to write information equivalent to level 10 (see below);
one file debug\_notroot.xx is open for all the other processes of
each model to write only error information if an error occurs.
\item {2} : one file debug.xx.xxxxxx is open by each process of each
model to write normal production diagnostics.
\item {5} : as for 2 with in addition some initial debug info.
\item {10} : as for 5 with in addition the routine calling tree.
\item {12} : as for 10 with in addition some routine calling notes.
\item {15} : as for 12 with even more debug diagnostics.
\item {20} : as for 15 with in addition some extra runtime analysis.
\item {30} : full debug information.
\end{itemize}
\subsection{Time statistics files}
\label{timestat}
The variable TIMER\_Debug, defined in the {\it namcouple} (second
number on the line below \$NLOGPRT keyword), is used to obtain time
statistics over all the processors for each routine.
Different output are written (in files named {\tt *.timers\_xxxx})
depending on TIMER\_Debug value :
\begin{itemize}
\item {TIMER\_Debug=0} : nothing is calculated, nothing is written.
\item {TIMER\_Debug=1} : the times are calculated and written in a
single file by the process 0 as well as the min and the max times
over all the processes.
\item {TIMER\_Debug=2} : the times are calculated and each process
writes its own file ; process 0 also writes the min and the max
times over all the processes in its file.
\item {TIMER\_Debug=3} : the times are calculated and each process
writes its own file ; process 0 also writes in its file the min
and the max times over all processes and also writes in its file
all the results for each process.
\end{itemize}
The time given for each timer is calculated by the difference between
calls to {\tt oasis\_timer\_start()} and {\tt oasis\_timer\_stop()}
and is the accumulated time over the entire run. Here is an overview
of the meaning of the different timers as implemented by default.
\begin{itemize}
\item {'total after init'} : total time of the simulation, implemented
in {\tt mod\_oasis\_method}, i.e. between the end of {\tt
oasis\_init\_comp} and the call to {\tt mpi\_barrier} and {\tt
mpi\_finalize} in routine {\tt oasis\_terminate}.
\item {'map definition'} : time spent in {\tt mct\_gsmap\_init} in
routine {\tt oasis\_def\_partition}; this routine defines the
patterns of communication between the source and target processes.
\item {'cpl\_setup'} :time spent in {\tt oasis\_coupler\_setup}, which
sets up additional coupling aspects related to {\tt
oasis\_method\_enddef}.
\item {'cpl\_smatrd'} : time spent in {\tt
oasis\_coupler\_smatreaddnc} in {\tt mod\_oasis\_coupler} (routine
{\tt oasis\_coupler\_setup}); this routine performs a distributed
read of a NetCDF SCRIP file and returns weights in a distributed
SparseMatrix, and calls {\tt mct\_sMatP\_Init} which initialises the
sparse matrix vector multiplication.
\item {'oasis\_advance\_init()'} : time spent in {\tt
oasis\_advance\_init}, which handles reading of initial coupling
restart and communication of data for fields with positive lags.
\item {'grcv\_00x'} : time spent in the reception of field x in {\tt
mct\_recv} (including communication and possible waiting time
linked to unbalance of components).
\item {'wout\_00x'} : time spent in the I/O for field x in routine
{\tt oasis\_advance\_run}.
\item {'gcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run}
in copying the field x just received in a local array.
\item {'pcpy\_00x'} : time spent in routine {\tt oasis\_advance\_run}
in copying the local field x in the array to send (i.e. with local
transformation besides division for averaging).
\item {'pavg\_00x'} : time spent in routine {\tt oasis\_advance\_run}
to calculate the average of field x (if done).
\item {'pmap\_00x'/'gmap\_00x'} : time spent in routine {\tt
oasis\_advance\_run} for the matrix vector multiplication for
field x on the source/target processes.
\item {'psnd\_00x'} : time spent in routine {\tt oasis\_advance\_run}
for sending field x (i.e. including call to {\tt mct\_waitsend} and
{\tt mct\_isend}).
\end{itemize}
|