1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
|
## ModelMetrics: Rapid Calculation of Model Metrics
[](https://travis-ci.org/JackStat/ModelMetrics)
[](https://ci.appveyor.com/project/JackStat/modelmetrics/branch/master)
[](https://coveralls.io/github/JackStat/ModelMetrics?branch=master)
[](https://CRAN.R-project.org/package=ModelMetrics)
Tyler Hunt thunt@snapfinance.com
### Introduction
ModelMetrics is a much faster and reliable package for evaluating models. ModelMetrics is written in using Rcpp making it faster than the other packages used for model metrics.
### Installation
You can install this package from CRAN:
```r
install.packages("ModelMetrics")
```
Or you can install the development version from Github with [devtools](https://github.com/hadley/devtools):
```r
devtools::install_github("JackStat/ModelMetrics")
```
### Benchmark and comparison
```r
N = 100000
Actual = as.numeric(runif(N) > .5)
Predicted = as.numeric(runif(N))
actual = Actual
predicted = Predicted
s1 <- system.time(a1 <- ModelMetrics::auc(Actual, Predicted))
s2 <- system.time(a2 <- Metrics::auc(Actual, Predicted))
# Warning message:
# In n_pos * n_neg : NAs produced by integer overflow
s3 <- system.time(a3 <- pROC::auc(Actual, Predicted))
s4 <- system.time(a4 <- MLmetrics::AUC(Predicted, Actual))
# Warning message:
# In n_pos * n_neg : NAs produced by integer overflow
s5 <- system.time({pp <- ROCR::prediction(Predicted, Actual); a5 <- ROCR::performance(pp, 'auc')})
data.frame(
package = c("ModelMetrics", "pROC", "ROCR")
,Time = c(s1[[3]],s3[[3]],s5[[3]])
)
# MLmetrics and Metrics could not calculate so they are dropped from time comparison
# package Time
# 1 ModelMetrics 0.030
# 2 pROC 50.359
# 3 ROCR 0.358
```
|
