1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
|
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/generateHyperParsEffect.R
\name{generateHyperParsEffectData}
\alias{generateHyperParsEffectData}
\title{Generate hyperparameter effect data.}
\usage{
generateHyperParsEffectData(
tune.result,
include.diagnostics = FALSE,
trafo = FALSE,
partial.dep = FALSE
)
}
\arguments{
\item{tune.result}{(\link{TuneResult} | \link{ResampleResult})\cr
Result of \link{tuneParams} (or \link{resample} ONLY when used
for nested cross-validation). The tuning result (or results if the
output is from nested cross-validation), also containing the
optimizer results. If nested CV output is passed, each element in the list
will be considered a separate run, and the data from each run will be
included in the dataframe within the returned \code{HyperParsEffectData}.}
\item{include.diagnostics}{(\code{logical(1)})\cr
Should diagnostic info (eol and error msg) be included?
Default is \code{FALSE}.}
\item{trafo}{(\code{logical(1)})\cr
Should the units of the hyperparameter path be converted to the
transformed scale? This is only useful when trafo was used to create the
path.
Default is \code{FALSE}.}
\item{partial.dep}{(\code{logical(1)})\cr
Should partial dependence be requested based on converting to reg task? This
sets a flag so that we know to use partial dependence downstream. This
should most likely be set to \code{TRUE} if 2 or more hyperparameters were
tuned simultaneously. Partial dependence should always be requested when
more than 2 hyperparameters were tuned simultaneously. Setting to
\code{TRUE} will cause \link{plotHyperParsEffect} to automatically
plot partial dependence when called downstream.
Default is \code{FALSE}.}
}
\value{
(\code{HyperParsEffectData})
Object containing the hyperparameter effects dataframe, the tuning
performance measures used, the hyperparameters used, a flag for including
diagnostic info, a flag for whether nested cv was used, a flag for whether
partial dependence should be generated, and the optimization algorithm used.
}
\description{
Generate cleaned hyperparameter effect data from a tuning result or from a
nested cross-validation tuning result. The object returned can be used for
custom visualization or passed downstream to an out of the box mlr method,
\link{plotHyperParsEffect}.
}
\examples{
\dontshow{ if (requireNamespace("kernlab")) \{ }
\dontrun{
# 3-fold cross validation
ps = makeParamSet(makeDiscreteParam("C", values = 2^(-4:4)))
ctrl = makeTuneControlGrid()
rdesc = makeResampleDesc("CV", iters = 3L)
res = tuneParams("classif.ksvm", task = pid.task, resampling = rdesc,
par.set = ps, control = ctrl)
data = generateHyperParsEffectData(res)
plt = plotHyperParsEffect(data, x = "C", y = "mmce.test.mean")
plt + ylab("Misclassification Error")
# nested cross validation
ps = makeParamSet(makeDiscreteParam("C", values = 2^(-4:4)))
ctrl = makeTuneControlGrid()
rdesc = makeResampleDesc("CV", iters = 3L)
lrn = makeTuneWrapper("classif.ksvm", control = ctrl,
resampling = rdesc, par.set = ps)
res = resample(lrn, task = pid.task, resampling = cv2,
extract = getTuneResult)
data = generateHyperParsEffectData(res)
plotHyperParsEffect(data, x = "C", y = "mmce.test.mean", plot.type = "line")
}
\dontshow{ \} }
}
|
