mlr_measures_ci_wald_cv {mlr3inferr} | R Documentation |
Cross-Validation CI
Description
Confidence intervals for cross-validation. The method is asymptotically exact for the so called Test Error as defined by Bayle et al. (2020). For the (expected) risk, the confidence intervals tend to be too liberal. This inference method can only be applied to decomposable losses.
Parameters
Those from MeasureAbstractCi
, as well as:
-
variance
::"all-pairs"
or"within-fold"
How to estimate the variance. The results tend to be very similar.
Super classes
mlr3::Measure
-> mlr3inferr::MeasureAbstractCi
-> MeasureCiWaldCV
Methods
Public methods
Inherited methods
Method new()
Creates a new instance of this R6 class.
Usage
MeasureCiWaldCV$new(measure)
Arguments
measure
(
Measure
orcharacter(1)
)
A measure of ID of a measure.
Method clone()
The objects of this class are cloneable with this method.
Usage
MeasureCiWaldCV$clone(deep = FALSE)
Arguments
deep
Whether to make a deep clone.
References
Bayle, Pierre, Bayle, Alexandre, Janson, Lucas, Mackey, Lester (2020). “Cross-validation confidence intervals for test error.” Advances in Neural Information Processing Systems, 33, 16339–16350.
Examples
m_waldcv = msr("ci.wald_cv", "classif.ce")
m_waldcv
rr = resample(tsk("sonar"), lrn("classif.featureless"), rsmp("cv"))
rr$aggregate(m_waldcv)