mlr_measures_abstract_ci {mlr3inferr}R Documentation

Abstract Class for Confidence Intervals

Description

Base class for confidence interval measures. See section Inheriting on how to add a new method.

Details

The aggregator of the wrapped measure is ignored, as the inheriting CI dictates how the point estimate is constructed. If a measure for which to calculate a CI has ⁠$obs_loss⁠ but also a ⁠$trafo⁠, (such as RMSE), the delta method is used to obtain confidence intervals.

Parameters

Inheriting

To define a new CI method, inherit from the abstract base class and implement the private method: ⁠ci: function(tbl: data.table, rr: ResampleResult, param_vals: named ⁠list()⁠) -> numeric(3)⁠ If requires_obs_loss is set to TRUE, tbl contains the columns loss, row_id and iteration, which are the pointwise loss, Otherwise, tbl contains the result of rr$score() with the name of the loss column set to "loss". the identifier of the observation and the resampling iteration. It should return a vector containing the estimate, lower and upper boundary in that order.

In case the confidence interval is not of the form ⁠(estimate, estimate - z * se, estimate + z * se)⁠ it is also necessary to implement the private method: ⁠.trafo: function(ci: numeric(3), measure: Measure) -> numeric(3)⁠ Which receives a confidence interval for a pointwise loss (e.g. squared-error) and transforms it according to the transformation measure$trafo (e.g. sqrt to go from mse to rmse).

Super class

mlr3::Measure -> MeasureAbstractCi

Public fields

resamplings

(character())
On which resampling classes this method can operate.

measure

(Measure)

Methods

Public methods

Inherited methods

Method new()

Creates a new instance of this R6 class.

Usage
MeasureAbstractCi$new(
  measure = NULL,
  param_set = ps(),
  packages = character(),
  resamplings,
  label,
  delta_method = FALSE,
  requires_obs_loss = TRUE
)
Arguments
measure

(Measure)
The measure for which to calculate a confidence interval. Must have ⁠$obs_loss⁠.

param_set

(ParamSet)
Set of hyperparameters.

packages

(character())
Set of required packages. A warning is signaled by the constructor if at least one of the packages is not installed, but loaded (not attached) later on-demand via requireNamespace().

resamplings

(character())
To which resampling classes this measure can be applied.

label

(character(1))
Label for the new instance.

delta_method

(logical(1))
Whether to use the delta method for measures (such RMSE) that have a trafo.

requires_obs_loss

(logical(1))
Whether the inference method requires a pointwise loss function.


Method aggregate()

Obtain a point estimate, as well as lower and upper CI boundary.

Usage
MeasureAbstractCi$aggregate(rr)
Arguments
rr

(ResampleResult)
The resample result.

Returns

named numeric(3)


Method clone()

The objects of this class are cloneable with this method.

Usage
MeasureAbstractCi$clone(deep = FALSE)
Arguments
deep

Whether to make a deep clone.


[Package mlr3inferr version 0.1.0 Index]