avseqmc {avseqmc} | R Documentation |
Anytime-valid sequential estimation of the p-value of a Monte-Carlo test
Description
avseqmc()
performs anytime-valid sequential estimation of the
p-value of a Monte-Carlo test as described in Stoepker and Castro (2024,
Definition 6). Subsequent references to equations and sections in this
section of the reference manual pertain to this paper. The sequential
p-value estimate is based on the construction of Definition 6 (i.e. through
the confidence sequence by Robbins (1970)).
For first-time usage, it may be helpful to follow the examples in the package vignette.
Usage
avseqmc(
sample_G,
epsilon = NULL,
stopcrit = list(type = "futility", param = 0.05),
min_samples = 0,
max_samples = max(1000, min_samples),
compute_lower = FALSE
)
Arguments
sample_G |
Either: a function (without arguments) that draws one (or a
batch of) zero/one samples from the distribution G as in Equation (5), where
the function returns a vector of zeroes and ones; or an object from class
'avseqmc_progress' containing earlier progress from anytime-valid estimation
of the p-value. The function |
epsilon |
The desired risk of overestimated significance. Ignored if
sample_G is an object of class |
stopcrit |
The desired stopping criterion. Can use one of the two pre-defined stopping criteria from Section 4.1 as follows (with respect to the notation used in that section):
|
min_samples |
Minimum number of Monte-Carlo samples before returning the current p-value estimate. Defaults to 0. |
max_samples |
Maximum number of Monte-Carlo samples before returning the
current p-value estimate. Defaults to |
compute_lower |
Boolean; if |
Value
An object of class avseqmc_progress
containing the progress of the
sequentially estimated p-value. The object is a list containing the
following elements:
-
$epsilon
: risk of overestimated significance used in the sequential estimation. -
$sample_G
: function that samples (batches) from the Monte-Carlo distribution $G^*(X)$ as in Equation (5). -
$ptilde
: sequence of sequential $p$-value estimates. The final value in this sequence is the most recent estimate of the $p$-value. -
$Ltilde
: sequence of lower bounds of the confidence sequence based on the construction by Robbins (1970). Contains NA values if these were not computed by default throughstopcrit = list("type"="futility","param"=...)
or requested usingcompute_lower=TRUE
. -
$n
: total number of samples drawn from the MC sampler. -
$S
: total number of ones observed from the MC sampler. -
$B
: sequence of number of ones observed at each sampling timepoint (which can be greater than 1 ifsample_G
samples in batches) -
$Bn
: sequence of number of samples drawn from MC sampler at each timepoint (which can be greater than 1 ifsample_G
samples in batches)
References
Stoepker, I. V., and R. M. Castro. 2024. Inference with Sequential Monte-Carlo Computation of p-Values: Fast and Valid Approaches. https://doi.org/10.48550/arXiv.2409.18908.
Robbins, H. (1970). Statistical Methods Related to the Law of the Iterated Logarithm. The Annals of Mathematical Statistics, 41(5):1397–1409. http://dx.doi.org/10.1214/aoms/1177696786
See Also
init_avseqmc_progress
which can be used if one wishes
to resume progress based on earlier reported p-values estimated by
Monte-Carlo simulation.
Examples
# Minimal example using defaults:
set.seed(123)
library(avseqmc)
G1 <- function(){runif(1) < 0.01} # A mock MC function to demonstrate functionality
R1 <- avseqmc(sample_G = G1, epsilon = 0.001)
print(R1)
# Minimal example to resuming earlier estimation:
G2 <- function(){runif(1) < 0.03}
R2a <- avseqmc(sample_G = G2, epsilon = 0.001)
print(R2a)
R2b <- avseqmc(R2a)
print(R2b)
# Using built-in convergence stopping time:
G3 <- function(){runif(1) < 0.04}
R3 <- avseqmc(sample_G = G3,
epsilon = 0.001,
stopcrit = list("type" = "convergence", param = c(1e-5, 100)))
# Batch sampling example (drawing batches of size 50)
G4 <- function(){runif(50) < 0.04}
R4 <- avseqmc(sample_G = G4, epsilon = 0.001)
print(R4)