1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
|
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/nearZeroVar.R
\name{nearZeroVar}
\alias{nearZeroVar}
\alias{nzv}
\alias{checkResamples}
\alias{checkConditionalX}
\title{Identification of near zero variance predictors}
\usage{
nearZeroVar(
x,
freqCut = 95/5,
uniqueCut = 10,
saveMetrics = FALSE,
names = FALSE,
foreach = FALSE,
allowParallel = TRUE
)
checkConditionalX(x, y)
checkResamples(index, x, y)
}
\arguments{
\item{x}{a numeric vector or matrix, or a data frame with all numeric data}
\item{freqCut}{the cutoff for the ratio of the most common value to the
second most common value}
\item{uniqueCut}{the cutoff for the percentage of distinct values out of the
number of total samples}
\item{saveMetrics}{a logical. If false, the positions of the zero- or
near-zero predictors is returned. If true, a data frame with predictor
information is returned.}
\item{names}{a logical. If false, column indexes are returned. If true,
column names are returned.}
\item{foreach}{should the \pkg{foreach} package be used for the
computations? If \code{TRUE}, less memory should be used.}
\item{allowParallel}{should the parallel processing via the \pkg{foreach}
package be used for the computations? If \code{TRUE}, more memory will be
used but execution time should be shorter.}
\item{y}{a factor vector with at least two levels}
\item{index}{a list. Each element corresponds to the training set samples in
\code{x} for a given resample}
}
\value{
For \code{nearZeroVar}: if \code{saveMetrics = FALSE}, a vector of
integers corresponding to the column positions of the problematic
predictors. If \code{saveMetrics = TRUE}, a data frame with columns:
\item{freqRatio }{the ratio of frequencies for the most common value over
the second most common value} \item{percentUnique }{the percentage of unique
data points out of the total number of data points} \item{zeroVar }{a vector
of logicals for whether the predictor has only one distinct value} \item{nzv
}{a vector of logicals for whether the predictor is a near zero variance
predictor}
For \code{checkResamples} or \code{checkConditionalX}, a vector of column
indicators for predictors with empty conditional distributions in at least
one class of \code{y}.
}
\description{
\code{nearZeroVar} diagnoses predictors that have one unique value (i.e. are
zero variance predictors) or predictors that are have both of the following
characteristics: they have very few unique values relative to the number of
samples and the ratio of the frequency of the most common value to the
frequency of the second most common value is large. \code{checkConditionalX}
looks at the distribution of the columns of \code{x} conditioned on the
levels of \code{y} and identifies columns of \code{x} that are sparse within
groups of \code{y}.
}
\details{
For example, an example of near zero variance predictor is one that, for
1000 samples, has two distinct values and 999 of them are a single value.
To be flagged, first the frequency of the most prevalent value over the
second most frequent value (called the ``frequency ratio'') must be above
\code{freqCut}. Secondly, the ``percent of unique values,'' the number of
unique values divided by the total number of samples (times 100), must also
be below \code{uniqueCut}.
In the above example, the frequency ratio is 999 and the unique value
percentage is 0.0001.
Checking the conditional distribution of \code{x} may be needed for some
models, such as naive Bayes where the conditional distributions should have
at least one data point within a class.
\code{nzv} is the original version of the function.
}
\examples{
nearZeroVar(iris[, -5], saveMetrics = TRUE)
data(BloodBrain)
nearZeroVar(bbbDescr)
nearZeroVar(bbbDescr, names = TRUE)
set.seed(1)
classes <- factor(rep(letters[1:3], each = 30))
x <- data.frame(x1 = rep(c(0, 1), 45),
x2 = c(rep(0, 10), rep(1, 80)))
lapply(x, table, y = classes)
checkConditionalX(x, classes)
folds <- createFolds(classes, k = 3, returnTrain = TRUE)
x$x3 <- x$x1
x$x3[folds[[1]]] <- 0
checkResamples(folds, x, classes)
}
\author{
Max Kuhn, with speed improvements to nearZeroVar by Allan Engelhardt
}
\keyword{utilities}
|
