metrics {nestedcv} | R Documentation |
Model performance metrics
Description
Returns model metrics from nestedcv models. Extended metrics including
Usage
metrics(object, extra = FALSE, innerCV = FALSE, positive = 2)
Arguments
object |
A 'nestcv.glmnet', 'nestcv.train', 'nestcv.SuperLearner' or 'outercv' object. |
extra |
Logical whether additional performance metrics are gathered for classification models: area under precision recall curve (PR.AUC, binary classification only), Cohen's kappa, F1 score, Matthews correlation coefficient (MCC). |
innerCV |
Whether to calculate metrics for inner CV folds. Only available for 'nestcv.glmnet' and 'nestcv.train' objects. |
positive |
For binary classification, either an integer 1 or 2 for the
level of response factor considered to be 'positive' or 'relevant', or a
character value for that factor. This affects the F1 score. See
|
Details
Area under precision recall curve is estimated by trapezoidal estimation
using MLmetrics::PRAUC()
.
For multi-class classification models, Matthews correlation coefficient is calculated using Gorodkin's method. Multi-class F1 score (macro F1) is calculated as the arithmetic mean of the class-wise F1 scores.
Value
A named numeric vector of performance metrics.
References
Gorodkin, J. (2004). Comparing two K-category assignments by a K-category correlation coefficient. Computational Biology and Chemistry. 28 (5): 367–374.