1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
---
title: "Developer workflows"
description: >
Learn about the workflows and conventions followed by arrow developers
output: rmarkdown::html_vignette
---
```{r setup-options, include=FALSE}
knitr::opts_chunk$set(error = TRUE, eval = FALSE)
```
The Arrow R package uses several additional development tools:
* [`air`](https://posit-dev.github.io/air/) for code styling
* [`lintr`](https://github.com/r-lib/lintr) for code analysis
* [`pkgdown`](https://pkgdown.r-lib.org) for building the website
* [`roxygen2`](https://roxygen2.r-lib.org) for documenting the package
- the R documentation uses the [`@examplesIf`](https://roxygen2.r-lib.org/articles/rd.html#functions) tag introduced in `roxygen2` version 7.1.2
Instructions for installing `air` can be found at <https://posit-dev.github.io/air/cli.html>.
You can install all the other additional dependencies by running:
```r
install.packages(c("lintr", "pkgdown", "roxygen2"))
```
The `arrow/r` directory contains a `Makefile` to help with some common tasks from the command line (e.g. `make test`, `make doc`, `make clean`, etc.).
## Loading arrow
You can load the R package via `devtools::load_all()`.
## Rebuilding the documentation
The R documentation uses the [`@examplesIf`](https://roxygen2.r-lib.org/articles/rd.html#functions) tag introduced in `{roxygen2}` version 7.1.2.
```r
remotes::install_github("r-lib/roxygen2")
```
You can use `devtools::document()` and `pkgdown::build_site()` to rebuild the documentation and preview the results.
```r
# Update roxygen documentation
devtools::document()
# To preview the documentation website
pkgdown::build_site(preview=TRUE)
```
## Styling and linting
Styling and linting can be set up and performed entirely with the [pre-commit](https://pre-commit.com/) tool:
```bash
pre-commit run --show-diff-on-failure --color=always --all-files r
```
See also the following subsections our styling and lint details for R and C++ codes.
### R code
The R code in the package follows [the tidyverse style](https://style.tidyverse.org/). On PR submission (and on pushes) our CI will run linting and will flag possible errors on the pull request with annotations.
You can automatically change the formatting of the code in the package using the [air](https://posit-dev.github.io/air/cli.html) formatter.
The `air` formatter will fix many styling errors, thought not all lintr errors are automatically fixable with `air`. The list of files we intentionally do not style is in the `exclude` field in `r/air.toml`.
Linting and styling with [pre-commit](https://pre-commit.com/) as described above is the best way to ensure your changes are being checked properly but you can also run the tools individually if you prefer, from the `arrow/r` directory of the repository.
From the command line, run `air`:
```
air format
```
In R, run `lintr`:
```r
lintr::lint_package()
```
Note: To run lintr, we require the `cyclocomp` package to be installed first.
### C++ code
The arrow package uses some customized tools on top of [cpp11](https://cpp11.r-lib.org/) to prepare its
C++ code in `src/`. This is because there are some features that are only enabled
and built conditionally during build time. If you change C++ code in the R
package, you will need to set the `ARROW_R_DEV` environment variable to `true`
(optionally, add it to your `~/.Renviron` file to persist across sessions) so
that the `data-raw/codegen.R` file is used for code generation. The `Makefile`
commands also handles this automatically.
We use Google C++ style in our C++ code. The easiest way to accomplish this is
use an editors/IDE that formats your code for you. Many popular editors/IDEs
have support for running `clang-format` on C++ files when you save them.
Installing/enabling the appropriate plugin may save you much frustration.
## Running tests
Tests can be run either using `devtools::test()` or the Makefile alternative.
```r
# Run the test suite, optionally filtering file names
devtools::test(filter="^regexp$")
```
```bash
# or the Makefile alternative from the arrow/r directory in a shell:
make test file=regexp
```
Some tests are conditionally enabled based on the availability of certain
features in the package build (S3 support, compression libraries, etc.).
Others are generally skipped by default but can be enabled with environment
variables or other settings:
* All tests are skipped on Linux if the package builds without the C++ libarrow.
To make the build fail if libarrow is not available (as in, to test that
the C++ build was successful), set `TEST_R_WITH_ARROW=true`
* Some tests are disabled unless `ARROW_R_DEV=true`
* Tests that require allocating >2GB of memory to test Large types are disabled
unless `ARROW_LARGE_MEMORY_TESTS=true`
* Integration tests against a real S3 bucket are disabled unless credentials
are set in `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`; these are available
on request
* S3 tests using [MinIO](https://min.io/) locally are enabled if the
`minio server` process is found running. If you're running MinIO with custom
settings, you can set `MINIO_ACCESS_KEY`, `MINIO_SECRET_KEY`, and
`MINIO_PORT` to override the defaults.
## Running checks
You can run package checks by using `devtools::check()` and check test coverage
with `covr::package_coverage()`.
```r
# All package checks
devtools::check()
# See test coverage statistics
covr::report()
covr::package_coverage()
```
For full package validation, you can run the following commands from a terminal.
```bash
R CMD build .
R CMD check arrow_*.tar.gz --as-cran
```
## Running extended CI checks
On a pull request, there are some actions you can trigger by commenting on the
PR. These extended CI checks are run nightly and can also be requested on-demand
using an internal tool called
[crossbow](https://arrow.apache.org/docs/developers/continuous_integration/crossbow.html).
A few important GitHub comment commands are shown below.
#### Run all extended R CI tasks
```
@github-actions crossbow submit -g r
```
This runs each of the R-related CI tasks.
#### Run a specific task
```
@github-actions crossbow submit {task-name}
```
See the `r:` group definition near the beginning of the [crossbow configuration](https://github.com/apache/arrow/blob/main/dev/tasks/tasks.yml)
for a list of glob expression patterns that match names of items in the `tasks:`
list below it.
|
