% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/bayesfactor_models.R
\name{bayesfactor_models}
\alias{bayesfactor_models}
\alias{bf_models}
\alias{bayesfactor_models.default}
\title{Bayes Factors (BF) for model comparison}
\usage{
bayesfactor_models(..., denominator = 1, verbose = TRUE)

bf_models(..., denominator = 1, verbose = TRUE)

\method{bayesfactor_models}{default}(..., denominator = 1, verbose = TRUE)
}
\arguments{
\item{...}{Fitted models (see details), all fit on the same data, or a single
\code{BFBayesFactor} object (see 'Details'). Ignored in \code{as.matrix()},
\code{update()}. If the following named arguments are present, they are passed
to \code{\link[insight:get_loglikelihood]{insight::get_loglikelihood()}} (see details):
\itemize{
\item \code{estimator} (defaults to \code{"ML"})
\item \code{check_response}  (defaults to \code{FALSE})
}}

\item{denominator}{Either an integer indicating which of the models to use as
the denominator, or a model to be used as a denominator. Ignored for
\code{BFBayesFactor}.}

\item{verbose}{Toggle off warnings.}
}
\value{
A data frame containing the models' formulas (reconstructed fixed and
random effects) and their \code{log(BF)}s  (Use \code{as.numeric()} to extract the
non-log Bayes factors; see examples), that prints nicely.
\cr\cr
For \code{as.matrix()} a square matrix of (log) Bayes factors, with rows as
denominators and columns as numerators.
}
\description{
This function computes or extracts Bayes factors from fitted
models.
\cr\cr
The \verb{bf_*} function is an alias of the main function.
\cr\cr
\strong{For more info, see \href{https://easystats.github.io/bayestestR/articles/bayes_factors.html}{the Bayes factors vignette}.}
}
\details{
If the passed models are supported by \strong{insight} the DV of all models will
be tested for equality (else this is assumed to be true), and the models'
terms will be extracted (allowing for follow-up analysis with \link{bayesfactor_inclusion}).
\itemize{
\item For \code{brmsfit} or \code{stanreg} models, Bayes factors are computed using the \CRANpkg{bridgesampling} package.
\itemize{
\item \code{brmsfit} models must have been fitted with \code{save_pars = save_pars(all = TRUE)}.
\item \code{stanreg} models must have been fitted with a defined \code{diagnostic_file}.
}
\item For \code{BFBayesFactor}, \code{bayesfactor_models()} is a wraparound \code{BayesFactor::extractBF()}.
\item For all other model types, Bayes factors are computed using the BIC approximation.
Note that BICs are extracted from using \link[insight:get_loglikelihood]{insight::get_loglikelihood}, see documentation
there for options for dealing with transformed responses and REML estimation.
}
\subsection{Additional methods}{

The resulting output is supported by the following methods:
\itemize{
\item \code{as.matrix()}: Extract a full matrix of (log-)Bayes factors between all
models (using the transitivity of Bayes factors).
\item \code{update()}: subset and/or re-reference the Bayes factors to a different model.
\item \code{as.numeric()}: Extract the (possibly log-)Bayes factor values.
}

See examples and \link{bayesfactor_methods}.
}
}
\note{
There is also a \href{https://easystats.github.io/see/articles/bayestestR.html}{\code{plot()}-method}
implemented in the \href{https://easystats.github.io/see/}{\pkg{see}-package}.
}
\section{Prior and posterior considerations}{

In order to correctly and precisely estimate Bayes factors, a rule of thumb
are the 4 P's: \strong{P}roper \strong{P}riors and \strong{P}lentiful
\strong{P}osteriors.
\cr\cr
For the computation of Bayes factors, the model priors must be proper priors
(at the very least they should be \emph{not flat}, and it is preferable that they
be \emph{informative}) (Note that by default, \code{brms::brm()} uses flat priors for
fixed-effects); Wide priors result in smaller marginal likelihoods, and thus
models with wider priors are trivially less likely than models with narrower
priors - where, at the extreme, that a model with completely flat priors is
infinitely less favorable than a point null model (this is called \emph{the
Jeffreys-Lindley-Bartlett paradox}). Thus, you should only ever try (or want)
to compute a Bayes factor when you have an informed prior.
\cr\cr
Additionally, for models using MCMC estimation the number of posterior
samples needed for testing is substantially larger than for estimation (the
default of 4000 samples may not be enough in many cases). A conservative rule
of thumb is to obtain 10 times more samples than would be required for
estimation (\emph{Gronau, Singmann, & Wagenmakers, 2017}). If less than 40,000
samples are detected, a warning is issued.
}

\section{Transitivity of Bayes factors}{

For multiple inputs (models or hypotheses), the function will return multiple
Bayes factors between each model and \emph{the same} reference model (the
\code{denominator} or un-restricted model). However, we can take advantage of the
transitivity of Bayes factors - where if we have two Bayes factors for Model
\emph{A} and model \emph{B} against the \emph{same reference model C}, we can obtain a Bayes
factor for comparing model \emph{A} to model \emph{B} by dividing them:
\cr\cr
\deqn{BF_{AB} = \frac{BF_{AC}}{BF_{BC}} = \frac{\frac{ML_{A}}{ML_{C}}}{\frac{ML_{B}}{ML_{C}}} = \frac{ML_{A}}{ML_{B}}}
\cr\cr
(Where \emph{ML} is the \emph{marginal likelihood}.)
\cr\cr
A full matrix comparing all models can be obtained with \code{as.matrix()}.
}

\section{Interpreting Bayes Factors}{

A Bayes factor greater than 1 can be interpreted as evidence against the
null, at which one convention is that a Bayes factor greater than 3 can be
considered as "substantial" evidence against the null (and vice versa, a
Bayes factor smaller than 1/3 indicates substantial evidence in favor of the
null-model). See also \code{effectsize::interpret_bf()}.
}

\examples{
\dontshow{if (require("lme4") && require("BayesFactor") && require("rstanarm") && require("brms")) withAutoprint(\{ # examplesIf}
# With lm objects:
# ----------------
lm1 <- lm(mpg ~ 1, data = mtcars)
lm2 <- lm(mpg ~ hp, data = mtcars)
lm3 <- lm(mpg ~ hp + drat, data = mtcars)
lm4 <- lm(mpg ~ hp * drat, data = mtcars)
(BFM <- bayesfactor_models(lm1, lm2, lm3, lm4, denominator = 1))
# bayesfactor_models(lm2, lm3, lm4, denominator = lm1) # same result
# bayesfactor_models(lm1, lm2, lm3, lm4, denominator = lm1) # same result

update(BFM, reference = "bottom")
as.matrix(BFM)
as.numeric(BFM)

lm2b <- lm(sqrt(mpg) ~ hp, data = mtcars)
# Set check_response = TRUE for transformed responses
bayesfactor_models(lm2b, denominator = lm2, check_response = TRUE)

\donttest{
# With lmerMod objects:
# ---------------------
lmer1 <- lme4::lmer(Sepal.Length ~ Petal.Length + (1 | Species), data = iris)
lmer2 <- lme4::lmer(Sepal.Length ~ Petal.Length + (Petal.Length | Species), data = iris)
lmer3 <- lme4::lmer(
  Sepal.Length ~ Petal.Length + (Petal.Length | Species) + (1 | Petal.Width),
  data = iris
)
bayesfactor_models(lmer1, lmer2, lmer3,
  denominator = 1,
  estimator = "REML"
)

# rstanarm models
# ---------------------
# (note that a unique diagnostic_file MUST be specified in order to work)
stan_m0 <- suppressWarnings(rstanarm::stan_glm(Sepal.Length ~ 1,
  data = iris,
  family = gaussian(),
  diagnostic_file = file.path(tempdir(), "df0.csv")
))
stan_m1 <- suppressWarnings(rstanarm::stan_glm(Sepal.Length ~ Species,
  data = iris,
  family = gaussian(),
  diagnostic_file = file.path(tempdir(), "df1.csv")
))
stan_m2 <- suppressWarnings(rstanarm::stan_glm(Sepal.Length ~ Species + Petal.Length,
  data = iris,
  family = gaussian(),
  diagnostic_file = file.path(tempdir(), "df2.csv")
))
bayesfactor_models(stan_m1, stan_m2, denominator = stan_m0, verbose = FALSE)


# brms models
# --------------------
# (note the save_pars MUST be set to save_pars(all = TRUE) in order to work)
brm1 <- brms::brm(Sepal.Length ~ 1, data = iris, save_pars = save_pars(all = TRUE))
brm2 <- brms::brm(Sepal.Length ~ Species, data = iris, save_pars = save_pars(all = TRUE))
brm3 <- brms::brm(
  Sepal.Length ~ Species + Petal.Length,
  data = iris,
  save_pars = save_pars(all = TRUE)
)

bayesfactor_models(brm1, brm2, brm3, denominator = 1, verbose = FALSE)


# BayesFactor
# ---------------------------
data(puzzles)
BF <- BayesFactor::anovaBF(RT ~ shape * color + ID,
  data = puzzles,
  whichRandom = "ID", progress = FALSE
)
BF
bayesfactor_models(BF) # basically the same
}
\dontshow{\}) # examplesIf}
}
\references{
\itemize{
\item Gronau, Q. F., Singmann, H., & Wagenmakers, E. J. (2017). Bridgesampling: An R package for estimating
normalizing constants. arXiv preprint arXiv:1710.08162.
\item Kass, R. E., and Raftery, A. E. (1995). Bayes Factors. Journal of the American Statistical Association,
90(430), 773-795.
\item Robert, C. P. (2016). The expected demise of the Bayes factor. Journal of Mathematical Psychology,
72, 33–37.
\item Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of p values.
Psychonomic bulletin & review, 14(5), 779-804.
\item Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., and Wagenmakers, E.-J. (2011).
Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests.
Perspectives on Psychological Science, 6(3), 291–298. \doi{10.1177/1745691611406923}
}
}
\seealso{
\code{\link[=bayesfactor_inclusion]{bayesfactor_inclusion()}} for testing predictors across Bayesian models.

Other Bayes factors: 
\code{\link{bayesfactor_inclusion}()},
\code{\link{bayesfactor_parameters}()},
\code{\link{bayesfactor_restricted}()}
}
\author{
Mattan S. Ben-Shachar
}
\concept{Bayes factors}
