File: index.md

package info (click to toggle)
gitlab 17.6.5-19
  • links: PTS, VCS
  • area: main
  • in suites: sid
  • size: 629,368 kB
  • sloc: ruby: 1,915,304; javascript: 557,307; sql: 60,639; xml: 6,509; sh: 4,567; makefile: 1,239; python: 406
file content (548 lines) | stat: -rw-r--r-- 18,483 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
---
stage: Verify
group: Runner
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---

# CI/CD steps

DETAILS:
**Tier:** Free, Premium, Ultimate
**Offering:** GitLab.com, Self-managed, GitLab Dedicated
**Status:** Experiment

Steps are reusable units of a job that when composed together replace the `script` used in a GitLab CI/CD job.
While you are not required to use steps, the reusability, composability, testability, and independence
of steps make it easier to understand and maintain CI/CD pipeline.

To get started, you can try the [Set up steps tutorial](../../tutorials/setup_steps/index.md).
To start creating your own steps, see [Creating your own step](#create-your-own-step).

This experimental feature is still in active development and might have breaking
changes at any time. Review the [changelog](https://gitlab.com/gitlab-org/step-runner/-/blob/main/CHANGELOG.md)
for full details on any breaking changes.

CI/CD steps are different than [CI/CD components](../components/index.md). Components
are reusable single pipeline configuration units. They are included in a pipeline when it is created,
adding jobs and configuration to the pipeline. Files such as common scripts or programs
from the component project cannot be referenced from a CI/CD job.

CI/CD Steps are reusable units of a job. When the job runs, the referenced step is downloaded to
the execution environment or image, bringing along any extra files included with the step.
Execution of the step replaces the `script` in the job.

## Step workflow

A step either runs a sequence of steps or executes a command. Each step specifies inputs received and outputs returned, has
access to CI/CD job variables, environment variables, and resources provided by the execution environment such as the file
system and networking. Steps are hosted locally on the file system, in GitLab.com repositories, or in any other Git source.

Additionally, steps:

- Run in a Docker container created by the Steps team, you can review the [`Dockerfile`](https://gitlab.com/gitlab-org/step-runner/-/blob/main/Dockerfile).
  Follow [epic 15073](https://gitlab.com/groups/gitlab-org/-/epics/15073) to track
  when steps will run inside the environment defined by the CI/CD job.
- Are specific to Linux. Follow [epic 15074](https://gitlab.com/groups/gitlab-org/-/epics/15074)
  to track when steps supports multiple operating systems.

For example, this job uses the [`run`](../yaml/index.md#run) CI/CD keyword to run a step:

```yaml
job:
  variables:
    CI_SAY_HI_TO: "Sally"
  run:
    - name: say_hi
      step: gitlab.com/gitlab-org/ci-cd/runner-tools/echo-step@v1.0.0
      inputs:
        message: "hello, ${{job.CI_SAY_HI_TO}}"
```

When this job runs, the message `hello, Sally` is printed to job log.
The definition of the echo step is:

```yaml
spec:
  inputs:
    message:
      type: string
---
exec:
  command:
    - bash
    - -c
    - echo '${{inputs.message}}'
```

## Use CI/CD Steps

Configure a GitLab CI/CD job to use CI Steps with the `run` keyword. You cannot use `before_script`,
`after_script`, or `script` in a job when you are running CI/CD Steps.

The `run` keyword accepts a list of steps to run. Steps are run one at a time in the order they are defined in the list.
Each list item has a `name` and either `step`, `script`, or `action`.

Name must consist only of alphanumeric characters and underscores, and must not start with a number.

### Run a step

Run a step by providing the [step location](#step-location) using the `step` keyword.

Inputs and environment variables can be passed to the step, and these can contain expressions that interpolate values.
Steps run in the directory defined by the `CI_BUILDS_DIR` [predefined variable](../variables/predefined_variables.md).

For example, the echo step loaded from the Git repository `gitlab.com/components/echo`
receives the environment variable `USER: Fred` and the input `message: hello Sally`:

```yaml
job:
  variables:
    CI_SAY_HI_TO: "Sally"
  run:
    - name: say_hi
      step: gitlab.com/components/echo@v1.0.0
      env:
        USER: "Fred"
      inputs:
        message: "hello ${{job.CI_SAY_HI_TO}}"
```

### Run a script

Run a script in a shell with the `script` keyword. Environment variables passed to scripts
using `env` are set in the shell. Script steps run in the directory defined by the `CI_BUILDS_DIR`
[predefined variable](../variables/predefined_variables.md).

For example, the following script prints the GitLab user to the job log:

```yaml
my-job:
  run:
    - name: say_hi
      script: echo hello ${{job.GITLAB_USER_LOGIN}}
```

Script steps always use the `bash` shell. Follow [issue 109](https://gitlab.com/gitlab-org/step-runner/-/issues/109)
to track when shell fallback is supported.

### Run a GitHub action

Run GitHub actions with the `action` keyword. Inputs and environment variables are passed directly to the
action, and action outputs are returned as step outputs. Action steps run in the directory
defined by the `CI_PROJECT_DIR` [predefined variable](../variables/predefined_variables.md).

Running actions requires the `dind` service. For more information, see
[Use Docker to build Docker images](../docker/using_docker_build.md).

For example, the following step uses `action` to make `yq` available:

```yaml
my-job:
  run:
    - name: say_hi_again
      action: mikefarah/yq@master
      inputs:
        cmd: echo ["hi ${{job.GITLAB_USER_LOGIN}} again!"] | yq .[0]
```

#### Known issues

Actions running in GitLab do not support uploading artifacts directly.
Artifacts must be written to the file system and cache instead, and selected with the
existing [`artifacts` keyword](../yaml/index.md#artifacts) and [`cache` keyword](../yaml/index.md#cache).

### Step location

Steps are loaded from a relative path on the file system, GitLab.com repositories,
or any other Git source.

#### Load a step from the file system

Load a step from the file system using a relative path that starts with a full-stop `.`.
The folder referenced by the path must contain a `step.yml` step definition file.
Path separators must always use forward-slashes `/`, regardless of operating system.

For example:

```yaml
- name: my-step
  step: ./path/to/my-step
```

#### Load a step from a Git repository

Load a step from a Git repository by supplying the URL and revision (commit, branch, or tag) of the repository.
You can also specify the relative directory of the step inside the repository.
If the URL is specified without a directory, then `step.yml` is loaded from the root folder of the repository.

For example:

- Specify the step with a branch:

  ```yaml
  job:
    run:
      - name: my_echo_step
        step: gitlab.com/gitlab-org/ci-cd/runner-tools/echo-step@main
  ```

- Specify the step with a tag:

  ```yaml
  job:
    run:
      - name: my_echo_step
        step: gitlab.com/gitlab-org/ci-cd/runner-tools/echo-step@v1.0.0
  ```

- Specify the step with a Git rev and directory in a repository:

  ```yaml
  job:
    run:
      - name: specifying_a_revision_and_directory_within_the_repository
        step:
          git:
            url: gitlab.com/gitlab-org/ci-cd/runner-tools/echo-step
            dir: reverse
            rev: main
  ```

Steps can't reference Git repositories using annotated tags. Follow [issue 123](https://gitlab.com/gitlab-org/step-runner/-/issues/123)
to track when annotated tags are supported.

### Expressions

Expressions are a mini-language enclosed in double curly-braces `${{ }}`. Expressions are evaluated
just prior to step execution in the job environment and can be used in:

- Input values
- Environment variable values
- Step location URL
- The executable command
- The executable work directory
- Outputs in a sequence of steps
- The `script` step
- The `action` step

Expressions can reference the following variables:

| Variable                    | Example                            | Description |
|:----------------------------|:-----------------------------------|:------------|
| `env`                       | `${{env.HOME}}`                    | Access to environment variables set on the execution environment or in previous steps. |
| `export_file`               | `echo name=FRED >${{export_file}}` | The path to the export file. Write to this file to export environment variables for use by subsequent running steps. |
| `inputs`                    | `${{inputs.message}}`              | Access inputs to the step. |
| `job`                       | `${{job.GITLAB_USER_NAME}}`        | Access GitLab CI/CD job variables, limited to those starting with `CI_`, `DOCKER_` or `GITLAB_`. |
| `output_file`               | `echo name=Fred >${{output_file}}` | The path to the output file. Write to this file to set output variables from the step. |
| `step_dir`                  | `work_dir: ${{step_dir}}`          | The folder to where the step has been downloaded. Use to refer to files in the step, or to set the work directory of an executable step. |
| `steps.[step-name].outputs` | `${{steps.my-step.outputs.name}}`  | Access to outputs from previously executed steps. Choose the specific step using the step name. |
| `work_dir`                  | `${{work_dir}}`                    | The work directory of an executing step. |

Expressions are different from template interpolation which uses double square-brackets (`$[[ ]]`)
and are evaluated during job generation.

Expressions only have access to CI/CD job variables with names starting with `CI_`, `DOCKER_`,
or `GITLAB_`. Follow [epic 15073](https://gitlab.com/groups/gitlab-org/-/epics/15073+)
to track when steps can access all CI/CD job variables.

### Using prior step outputs

Step inputs can reference outputs from prior steps by referencing the step name and output variable name.

For example, if the `gitlab.com/components/random-string` step defined an output variable called `random_value`:

```yaml
job:
  run:
    - name: generate_rand
      step: gitlab.com/components/random
    - name: echo_random
      step: gitlab.com/components/echo
      inputs:
        message: "The random value is: ${{steps.generate_rand.random_value}}"
```

### Environment variables

Steps can [set](#set-environment-variables) environment variables, [export](#export-an-environment-variable)
environment variables, and environment variables can be passed in when using `step`, `script`, or `action`.

Environment variable precedence, from highest to lowest precedence, are variables set:

1. By using `env` keyword in the `step.yml`.
1. By using the `env` keyword passed to a step in a sequence of steps.
1. By using the `env` keyword for all steps in a sequence.
1. Where a previously run step has written to `${{export_file}}`.
1. By the Runner.
1. By the container.

## Create your own step

Create your own step by performing the following tasks:

1. Create a GitLab project, a Git repository, or a directory on a file system that is accessible
   when the CI/CD job runs.
1. Create a `step.yml` file and place it in the root folder of the project, repository, or directory.
1. Define the [specification](#the-step-specification) for the step in the `step.yml`.
1. Define the [definition](#the-step-definition) for the step in the `step.yml`.
1. Add any files that your step uses to the project, repository, or directory.

After the step is created, you can [use the step in a job](#run-a-step).

### The step specification

The step specification is the first of two documents contained in the step `step.yml`.
The specification defines inputs and outputs that the step receives and returns.

#### Specify inputs

Input names can only use alphanumeric characters and underscores, and must not start with a number.
Inputs must have a type, and they can optionally specify a default value. An input with no default value
is a required input, it must be specified when using the step.

Inputs must be one of the following types.

| Type      | Example                 | Description |
|:----------|:------------------------|:------------|
| `array`   | `["a","b"]`             | A list of un-typed items. |
| `boolean` | `true`                  | True or false. |
| `number`  | `56.77`                 | 64 bit float. |
| `string`  | `"brown cow"`           | Text.       |
| `struct`  | `{"k1":"v1","k2":"v2"}` | Structured content. |

For example, to specify that the step accepts an optional input called `greeting` of type `string`:

```yaml
spec:
  inputs:
    greeting:
      type: string
      default: "hello, world"
---
```

To provide the input when using the step:

```yaml
run:
  - name: my_step
    step: ./my-step
    inputs:
      greeting: "hello, another world"
```

#### Specify outputs

Similar to inputs, output names can only use alphanumeric characters and underscores,
and must not start with a number. Outputs must have a type, and they can optionally specify a default value.
The default value is returned when the step doesn't return the output.

Outputs must be one of the following types.

| Type         | Example                 | Description |
|:-------------|:------------------------|:------------|
| `array`      | `["a","b"]`             | A list of un-typed items. |
| `boolean`    | `true`                  | True or false. |
| `number`     | `56.77`                 | 64 bit float. |
| `string`     | `"brown cow"`           | Text.       |
| `struct`     | `{"k1":"v1","k2":"v2"}` | Structured content. |

For example, to specify that the step returns an output called `value` of type `number`:

```yaml
spec:
  outputs:
    value:
      type: number
---
```

To use the output when using the step:

```yaml
run:
  - name: random_generator
    step: ./random-generator
  - name: echo_number
    step: ./echo
    inputs:
      message: "Random number generated was ${{step.random-generator.outputs.value}}"
```

#### Specify delegated outputs

Instead of specifying output names and types, outputs can be entirely delegated to a sub-step.
The outputs returned by the sub-step are returned by your step. The `delegate` keyword
in the step definition determines which sub-step outputs are returned by the step.

For example, the following step returns outputs returned by the `random-generator`.

```yaml
spec:
  outputs: delegate
---
steps:
  - name: random_generator
    step: ./random-generator
delegate: random-generator
```

#### Specify no inputs or outputs

A step might not require any inputs or return any outputs. This could be when a step
only writes to disk, sets an environment variable, or prints to STDOUT. In this case,
`spec:` is empty:

```yaml
spec:
---
```

### The step definition

Steps can:

- Set environment variables
- Execute a command
- Run a sequence of other steps.

#### Set environment variables

Set environment variables by using the `env` keyword. Environment variable names can only use
alphanumeric characters and underscores, and must not start with a number.

Environment variables are made available either to the executable command or to all of the steps
if running a sequence of steps. For example:

```yaml
spec:
---
env:
  FIRST_NAME: Sally
  LAST_NAME: Seashells
steps:
  # omitted for brevity
```

Steps only have access to a subset of environment variables from the runner environment.
Follow [epic 15073](https://gitlab.com/groups/gitlab-org/-/epics/15073+) to track
when steps can access all environment variables.

#### Execute a command

A step declares it executes a command by using the `exec` keyword. The command must be specified,
but the working directory (`work_dir`) is optional. Environment variables set by the step
are available to the running process.

For example, the following step prints the step directory to the job log:

```yaml
spec:
---
exec:
  work_dir: ${{step_dir}}
  command:
    - bash
    - -c
    - "echo ${PWD}"
```

NOTE:
Any dependency required by the executing step should also be installed by the step.
For example, if a step calls `go`, it should first install it.

##### Return an output

Executable steps return an output by adding a line to the `${{output_file}}` in the format `name=value`,
where the value is a JSON representation of the output type. The type of value written by the step
must match the type of the output in the step specification.

For example, to return the output named `car` with `string` value `range rover`:

```yaml
spec:
  outputs:
    car:
      type: string
---
exec:
  command:
    - bash
    - -c
    - echo car=\"range rover\" >>${{output_file}}
```

##### Export an environment variable

Executable steps export an environment variable by adding a line to the `${{export_file}}`
in the format `name=value`. Double quotation marks are not required around the value.

For example, to set the variable `GOPATH` to value `/go`:

```yaml
spec:
---
exec:
  command:
    - bash
    - -c
    - echo GOPATH=/go >${{export_file}}
```

#### Run a sequence of steps

A step declares it runs a sequence of steps using the `steps` keyword. Steps run one at a time
in the order they are defined in the list. This syntax is the same as the `run` keyword.

Steps must have a name consisting only of alphanumeric characters and underscores, and must not start with a number.

For example, this step installs Go, then runs a second step that expects Go to already
have been installed:

```yaml
spec:
---
steps:
  - name: install_go
    step: ./go-steps/install-go
    inputs:
      version: "1.22"
  - name: format_go_code
    step: ./go-steps/go-fmt
    inputs:
      code: path/to/go-code
```

##### Return an output

Outputs are returned from a sequence of steps by using the `outputs` keyword.
The type of value in the output must match the type of the output in the step specification.

For example, the following step returns the installed Java version as an output.
This assumes the `install_java` step returns an output named `java_version`.

```yaml
spec:
  outputs:
    java_version:
      type: string
---
steps:
  - name: install_java
    step: ./common/install-java
outputs:
  java_version: "the java version is ${{steps.install_java.outputs.java_version}}"
```

Alternatively, all outputs of a sub-step can be returned using the `delegate` keyword.
For example:

```yaml
spec:
  outputs: delegate
---
steps:
  - name: install_java
    step: ./common/install-java
delegate: install_java
```