% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/api-estimation.R
\name{compare_mfrm}
\alias{compare_mfrm}
\title{Compare two or more fitted MFRM models}
\usage{
compare_mfrm(..., labels = NULL, warn_constraints = TRUE, nested = FALSE)
}
\arguments{
\item{...}{Two or more \code{mfrm_fit} objects to compare.}

\item{labels}{Optional character vector of labels for each model.
If \code{NULL}, labels are generated from model/method combinations.}

\item{warn_constraints}{Logical. If \code{TRUE} (the default), emit a warning
when models use different centering constraints (\code{noncenter_facet} or
\code{dummy_facets}), which can make information-criterion comparisons
misleading.}

\item{nested}{Logical. Set to \code{TRUE} only when the supplied models are
known to be nested and fitted with the same likelihood basis on the same
observations. The default is \code{FALSE}, in which case no likelihood-ratio
test is reported. When \code{TRUE}, the function still runs a conservative
structural nesting review and computes the LRT only for supported nesting patterns.}
}
\value{
An object of class \code{mfrm_comparison} (named list) with:
\itemize{
\item \code{table}: data.frame of model-level statistics (LogLik, AIC, BIC,
Delta_AIC, AkaikeWeight, Delta_BIC, BICWeight, npar, nobs, WeightedN,
ICSampleSize, ICSampleSizeBasis, Model, Method, Converged, ICComparable).
\item \code{lrt}: data.frame with likelihood-ratio test result (only when two models
are supplied and \code{nested = TRUE}). Contains \code{ChiSq}, \code{df}, \code{p_value}.
\item \code{evidence_ratios}: data.frame of pairwise Akaike-weight ratios (Model1,
Model2, EvidenceRatio). \code{NULL} when weights cannot be computed.
\item \code{preferred}: named list with the preferred model label by each criterion.
\item \code{comparison_basis}: list describing whether IC and LRT comparisons were
considered comparable. Includes a conservative \code{nesting_review} plus
\code{lrt_status} / \code{lrt_reason} so withheld LRTs are explicit rather than
silently absent.
}
}
\description{
Produce a side-by-side comparison of multiple \code{\link[=fit_mfrm]{fit_mfrm()}} results using
information criteria, log-likelihood, and parameter counts. When exactly
two models are supplied and the current conservative nesting review passes,
a likelihood-ratio test is included.
}
\details{
Models should be fit to the \strong{same data} (same rows, same person/facet
columns) for the comparison to be meaningful. The function checks that
observation counts match and warns otherwise.

Information-criterion ranking is reported only when all candidate models
use the package's \code{MML} estimation path, analyze the same observations, and
converge successfully. Raw \code{AIC} and \code{BIC} values are still shown for each
model, but \verb{Delta_*}, weights, and preferred-model summaries are suppressed
when the likelihood basis is not comparable enough for primary reporting.
The comparison table records both row count (\code{nobs}) and the sample-size
basis used for the BIC penalty (\code{ICSampleSize}, \code{ICSampleSizeBasis}); for
weighted fits this is the sum of weights rather than the number of rows.

\strong{Nesting}: Two models are \emph{nested} when one is a special case of the
other obtained by imposing equality constraints.  The most common
nesting in MFRM is RSM (shared thresholds) inside PCM
(item-specific thresholds).  Models that differ only in estimation
method (MML vs JML) on the same specification are not nested in the
usual sense---use information criteria rather than LRT for that
comparison.

In the \strong{current \code{mfrmr} model space}, the automatic nesting review is
intentionally conservative. It currently supports two fixed-effect
restrictions under shared data and shared constraints:
\itemize{
\item \code{RSM} nested inside \code{PCM} when the \code{PCM} fit has an explicit
\code{step_facet};
\item same-family additive-vs-interaction comparisons when the smaller fit's
\code{facet_interactions} set is a subset of the larger fit's set.
}

Cross-method comparisons, comparisons that change anchors/dummying/centering,
and same-family comparisons that do not add fixed interaction terms are not
automatically promoted to LRT claims.

The \strong{likelihood-ratio test (LRT)} is reported only when exactly two
models are supplied, \code{nested = TRUE}, the structural nesting review passes, and the
difference in the number of parameters is positive:

\deqn{\Lambda = -2 (\ell_{\mathrm{restricted}} - \ell_{\mathrm{full}})
  \sim \chi^2_{\Delta p}}

The LRT is asymptotically valid when models are nested and the data
are independent.  With small samples or boundary conditions (e.g.,
variance components near zero), treat p-values as approximate.
}
\section{Information-criterion diagnostics}{

In addition to raw AIC and BIC values, the function computes:
\itemize{
\item \strong{Delta_AIC / Delta_BIC}: difference from the best (minimum) value.
A Delta < 2 is typically considered negligible; 4--7 suggests
moderate evidence; > 10 indicates strong evidence against the
higher-scoring model (Burnham & Anderson, 2002).
\item \strong{AkaikeWeight / BICWeight}: model probabilities derived from
\code{exp(-0.5 * Delta)}, normalised across the candidate set.  An
Akaike weight of 0.90 means the model has a 90\\% probability of
being the best in the candidate set.
\item \strong{Evidence ratios}: pairwise ratios of Akaike weights, quantifying
the relative evidence for one model over another (e.g., an
evidence ratio of 5 means the preferred model is 5 times more
likely).
}

AIC penalises complexity less than BIC; when they disagree, AIC
favours the more complex model and BIC the simpler one.
}

\section{What this comparison means}{

\code{compare_mfrm()} is a same-basis model-comparison helper. Its strongest
claims apply only when the models were fit to the same response data,
under a compatible likelihood basis, and with compatible constraint
structure.
}

\section{What this comparison does not justify}{

\itemize{
\item Do not treat AIC/BIC differences as primary evidence when
\code{table$ICComparable} is \code{FALSE}.
\item Do not interpret the LRT unless \code{nested = TRUE} and the structural nesting review
in \code{comparison_basis$nesting_review} passes.
\item Same-family additive-vs-interaction fits are considered nested only when
all other structural settings match and the smaller model's
\code{facet_interactions} set is a subset of the larger model's set.
\item Do not assume that \code{nested = TRUE} overrides the package's conservative
nesting boundary; unsupported relations remain unsupported.
\item Do not compare models fit to different datasets, different score codings,
or materially different constraint systems as if they were commensurate.
}
}

\section{Interpreting output}{

\itemize{
\item Lower AIC/BIC values indicate better parsimony-accuracy trade-off only
when \code{table$ICComparable} is \code{TRUE}.
\item A significant LRT p-value suggests the more complex model provides a
meaningfully better fit only when the nesting assumption truly holds.
\item \code{preferred} indicates the model preferred by each criterion.
\item \code{evidence_ratios} gives pairwise Akaike-weight ratios (returned only
when Akaike weights can be computed for at least two models).
\item When comparing more than two models, interpret evidence ratios
cautiously---they do not adjust for multiple comparisons.
}
}

\section{How to read the main outputs}{

\itemize{
\item \code{table}: first-pass comparison table; start with \code{ICComparable},
\code{Model}, \code{Method}, \code{AIC}, and \code{BIC}.
\item \code{comparison_basis}: records whether IC and LRT claims are defensible for
the supplied models. Inspect \code{comparison_basis$nesting_review$relation}
and \code{reason} before reading any LRT output.
\item \code{lrt}: nested-model test summary, present only when the requested and
reviewed conditions are met.
\item \code{preferred}: candidate preferred by each criterion when those summaries
are available.
}
}

\section{Recommended next step}{

Inspect \code{comparison_basis} before writing conclusions. If comparability is
weak, treat the result as descriptive and revise the model setup (for
example, explicit \code{step_facet}, common data, or common constraints) before
using IC or LRT results in reporting.
}

\section{Typical workflow}{

\enumerate{
\item Fit two models with \code{\link[=fit_mfrm]{fit_mfrm()}} (e.g., RSM and PCM).
\item Compare with \code{compare_mfrm(fit_rsm, fit_pcm)}.
\item Inspect \code{summary(comparison)} for AIC/BIC diagnostics and, when
appropriate, an LRT.
}
}

\section{References}{

\itemize{
\item Burnham, K. P., & Anderson, D. R. (2002). \emph{Model selection and
multimodel inference: A practical information-theoretic
approach} (2nd ed.). Springer.
\item Akaike, H. (1974). A new look at the statistical model
identification. \emph{IEEE Transactions on Automatic Control,
19}(6), 716-723.
\item Schwarz, G. (1978). Estimating the dimension of a model.
\emph{Annals of Statistics, 6}(2), 461-464.
}
}

\examples{
\donttest{
toy <- load_mfrmr_data("example_core")

fit_rsm <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score",
                     method = "MML", model = "RSM", quad_points = 7, maxit = 30)
fit_pcm <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score",
                     method = "MML", model = "PCM",
                     step_facet = "Criterion", quad_points = 7, maxit = 30)
comp <- compare_mfrm(fit_rsm, fit_pcm, labels = c("RSM", "PCM"))
comp$table
comp$evidence_ratios
}

}
\seealso{
\code{\link[=fit_mfrm]{fit_mfrm()}}, \code{\link[=diagnose_mfrm]{diagnose_mfrm()}}
}
