Skip to contents

Create a train-test evaluator

Source: R/eval_traintest.R
eval_traintest.Rd

Create a train-test evaluator

Usage

eval_traintest(
  split = 0.8,
  ntrials = 1,
  conf_type = c("norm", "perc"),
  contrasts = TRUE
)

Arguments

split

double between 0 and 1. percent of data to use as train set

ntrials

integer. number of train-test trials to run

conf_type

string. How to calculate confidence interval of performance metrics across trials: 'norm' calcualtes std err using the 'sd' function, 'perc' calculats lower and upper conf values using the 'quantile' function.

contrasts

logical. Whether to compare test performance of fits within each group-outcome-stat combination (i.e., between predictors). This will result in a p-value for each model comparison as the proporiton of trials where one model had a lower performance than another model. Thus, a p-value of 0.05 indicates that one model performed worse than the other model 5% of the trials. If ntrials == 1, then this value can only be 0 or 1 to indicate which model is better.

Value

an aba model with modified evals parameter

Examples

data <- adnimerge %>% dplyr::filter(VISCODE == 'bl')
model <- aba_model() %>%
  set_data(data) %>%
  set_groups(everyone()) %>%
  set_outcomes(ConvertedToAlzheimers, CSF_ABETA_STATUS_bl) %>%
  set_predictors(
    PLASMA_ABETA_bl, PLASMA_PTAU181_bl, PLASMA_NFL_bl,
    c(PLASMA_ABETA_bl, PLASMA_PTAU181_bl, PLASMA_NFL_bl)
  ) %>%
  set_stats('glm') %>%
  set_evals('traintest') %>%
  fit()
#> [1] "ConvertedToAlzheimers ~ PLASMA_ABETA_bl"
#> [1] "ConvertedToAlzheimers ~ PLASMA_PTAU181_bl"
#> [1] "ConvertedToAlzheimers ~ PLASMA_NFL_bl"
#> [1] "ConvertedToAlzheimers ~ PLASMA_ABETA_bl + PLASMA_PTAU181_bl + PLASMA_NFL_bl"
#> [1] "CSF_ABETA_STATUS_bl ~ PLASMA_ABETA_bl"
#> [1] "CSF_ABETA_STATUS_bl ~ PLASMA_PTAU181_bl"
#> [1] "CSF_ABETA_STATUS_bl ~ PLASMA_NFL_bl"
#> [1] "CSF_ABETA_STATUS_bl ~ PLASMA_ABETA_bl + PLASMA_PTAU181_bl + PLASMA_NFL_bl"