apm_est {apm} | R Documentation |
Estimate ATTs from models fits
Description
apm_est()
computes the ATTs from the models previously fit by apm_pre()
, choosing the optimal one by minimizing the largest absolute average prediction error across validation times. Optionally, this process can be simulated to arrive at a distribution of ATTs that accounts for the uncertainty in selecting the optimal model. plot()
plots the resulting ATT(s).
Usage
apm_est(
fits,
post_time,
M = 0,
R = 1000L,
all_models = FALSE,
cl = NULL,
verbose = TRUE,
...
)
## S3 method for class 'apm_est'
summary(object, level = 0.95, M = NULL, ...)
## S3 method for class 'apm_est'
plot(x, label = TRUE, size.weights = TRUE, ...)
Arguments
fits |
an |
post_time |
the value of the time variable considered post-treatment, for which the ATT is to be estimated. |
M |
the sensitivity parameter for set identification. For |
R |
the number of bootstrap iterations used to compute the sampling variance of the ATT. Default is 1000. More is better but takes longer. |
all_models |
|
cl |
a cluster object created by |
verbose |
|
... |
other arguments passed to |
level |
the desired confidence level. Set to 0 to ignore sampling variation in computing the interval bounds. Default is .95. |
x , object |
an |
label |
|
size.weights |
|
Details
apm_est()
estimates the ATT from each model and combines them to form the BMA-weighted estimate of the ATT. Uncertainty for the BMA-weighted ATT is computed by combining two variance components, one that account for sampling and another that accounts for model selection. The component due to sampling is computed by bootstrapping the process of fitting the outcome model for the post-treatment outcome identified by post_time
and computing the difference between the observed outcome mean difference and the model-predicted outcome mean difference. The fractional weighted bootstrap as implemented in fwb::fwb()
is used to ensure no units are dropped from the analysis. In each bootstrap sample, the BMA-weighted ATT estimate is computed as the weighted average of the ATTs computed from the models using the fixed BMA weights computed by apm_pre()
, and the variance is computed as the empirical variance over the bootstrapped estimates. The variance component due to model selection is computed as the BMA-weighted variance of the original ATTs.
When M
is greater than 0, bounds for set identification and their uncertainty are additionally computed. This involves bootstrapping the fitting of the pre-period models along with post-treatment models on order to compute the maximum absolute difference in average prediction errors for each model across validation periods. Each bootstrap sample produces a margin of error for each model computed as M \times \delta_m
where \delta_m
is the maximum absolute difference in average prediction errors for model m
. Upper and lower bounds for the set-identified BMA-weighted ATT are computed as \text{ATT}_m \pm M \times \delta_m
. The same procedure as above is then used to compute the variance of these bounds.
summary()
displays the BMA-weighted ATT estimate, its standard error, and Wald confidence intervals. When M
is greater than 0, bounds for the set-identified ATT are displayed in the confidence interval bound columns. The lower bound is computed as \text{LB} - \sigma_{LB}Z_{l}
and the upper bound as \text{UB} + \sigma_{UB}Z_{l}
, where \text{LB}
and \text{UB}
are the lower and upper bounds, \sigma_{LB}
and \sigma_{UB}
are their variances accounting for sampling and model selection, and Z_{l}
is the critical Z-statistic for confidence level l
. To display the set-identification bounds themselves, one should set level = 0
.
Value
apm_est()
returns an apm_est
object, which contains the ATT estimates and their variance estimates. The following components are included:
- BMA_att
the BMA-weighted ATT
- atts
a 1-column matrix containing the ATT estimates from each model (when
all_models = FALSE
, only models with positive BMA weights are included)- BMA_var
the total variance estimate for the BMA-weighted ATT incorporating the variance due to sampling and due to model selection
- BMA_var_b
the bootstrap-based component of the variance estimate for the BMA-weighted ATT due to sampling
- BMA_var_m
the component of the variance estimate for the BMA-weighted ATT due to model selection
- M
the value of the sensitivity parameter
M
- post_time
the value supplied to
post_time
- observed_means
a matrix of the observed outcome means at each pre-treatment validation period
- pred_errors
an array containing the average prediction errors for each model and each pre-treatment validation period
- pred_error_diffs
a matrix containing the difference in average prediction errors between groups for each model and each pre-treatment validation period
- BMA_weights
the BMA weights computed by
apm_pre()
(whenall_models = FALSE
, only positive BMA weights are included)- boot_out
an
fwb
object containing the bootstrap results
plot()
returns a ggplot
object displaying the ATT for each model plotted against the maximum absolute difference in average prediction errors for that model. The model with the lowest maximum absolute difference in average prediction errors is displayed in red.
summary()
produces a table with the BMA-weighted ATT, it's estimated standard error, and confidence interval limits. When M
is greater than 0, additional rows for each value of M
are included with the lower and upper bound. When level
is greater than 0, these bounds include the uncertainty due to sampling and model selection; otherwise, they correspond to the set identification bounds for the ATT.
See Also
apm_pre()
for computing the BMA weights; fwb::fwb()
for the fractional weighted bootstrap.
Examples
data("ptpdata")
# Combination of 4 models: 2 time trends, 2 lags
models <- apm_mod(list(crude_rate ~ 1),
lag = 0:1,
time_trend = 0:1)
models
# Fit the models to data; unit_var must be supplied for
# fixed effects
fits <- apm_pre(models,
data = ptpdata,
group_var = "group",
time_var = "year",
val_times = 2004:2007,
unit_var = "state",
nsim = 100,
verbose = FALSE)
est <- apm_est(fits,
post_time = 2008,
M = 1,
R = 20,
verbose = FALSE)
est
# ATT estimate and bounds for M = 1
summary(est)
# Bounds for other values of M
summary(est, M = c(.5, 1, 1.5, 2))
# Set-ID bounds without uncertainty
summary(est, level = 0)
plot(est)