1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
|
.. include:: ../../global.inc
.. include:: manual_chapter_numbers.inc
.. index::
pair: deprecated @files_re; Tutorial
.. _new_manual.deprecated_files_re:
#####################################################################################################################
|new_manual.deprecated_files_re.chapter_num|: **@files_re**: Deprecated `syntax using regular expressions`
#####################################################################################################################
.. warning ::
-
**This is deprecated syntax**
**which is no longer supported and**
**should NOT be used in new code.**
.. seealso::
* :ref:`Manual Table of Contents <new_manual.table_of_contents>`
* :ref:`decorators <decorators>`
* :ref:`@files_re <decorators.files_re>` syntax in detail
***************************************
Overview
***************************************
**@files_re** combines the functionality of @transform, @collate and @merge in
one overloaded decorator.
This is the reason why its use is discouraged. **@files_re** syntax is far too overloaded
and context-dependent to support its myriad of different functions.
The following documentation is provided to help maintain historical *Ruffus* usage.
=======================================
Transforming input and output filenames
=======================================
For example, the following code takes files from
the previous pipeline task, and makes new output parameters with the ``.sums`` suffix
in place of the ``.chunks`` suffix:
::
@transform(step_4_split_numbers_into_chunks, suffix(".chunks"), ".sums")
def step_5_calculate_sum_of_squares (input_file_name, output_file_name):
#
# calculate sums and sums of squares for all values in the input_file_name
# writing to output_file_name
""
This can be written using @files_re equivalently:
::
@files_re(step_4_split_numbers_into_chunks, r".chunks", r".sums")
def step_5_calculate_sum_of_squares (input_file_name, output_file_name):
""
.. _new_manual.files_re.combine:
.. index::
pair: combine; Manual
=====================================================
Collating many *inputs* into a single *output*
=====================================================
Similarly, the following code collects **inputs**
from the same species in the same directory:
::
@collate('*.animals', # inputs = all *.animal files
regex(r'mammals.([^.]+)'), # regular expression
r'\1/animals.in_my_zoo', # single output file per species
r'\1' ) # species name
def capture_mammals(infiles, outfile, species):
# summarise all animals of this species
""
This can be written using @files_re equivalently using the :ref:`combine<decorators.combine>` indicator:
::
@files_re('*.animals', # inputs = all *.animal files
r'mammals.([^.]+)', # regular expression
combine(r'\1/animals.in_my_zoo'), # single output file per species
r'\1' ) # species name
def capture_mammals(infiles, outfile, species):
# summarise all animals of this species
""
==============================================================================
Generating *input* and *output* parameter using regular expresssions
==============================================================================
The following code generates additional
*input* prerequisite file names which match the original *input* files.
We want each job of our ``analyse()`` function to get corresponding pairs
of ``xx.chunks`` and ``xx.red_indian`` files when
``*.chunks`` are generated by the task function ``split_up_problem()`` and
``*.red_indian`` are generated by the task function ``make_red_indians()``:
::
@follows(make_red_indians)
@transform(split_up_problem, # starting set of *inputs*
regex(r"(.*).chunks"), # regular expression
inputs([r"\g<0>", # xx.chunks
r"\1.red_indian"]), # important.file
r"\1.results" # xx.results
)
def analyse(input_filenames, output_file_name):
"Do analysis here"
The equivalent code using @files_re looks very similar:
::
@follows(make_red_indians)
@files_re( split_up_problem, # starting set of *inputs*
r"(.*).chunks", # regular expression
[r"\g<0>", # xx.chunks
r"\1.red_indian"]), # important.file
r"\1.results") # xx.results
def analyse(input_filenames, output_file_name):
"Do analysis here"
|
