Multi-group tests of profile similarity with mglpa

mglpa implements, entirely in R, the sequence of multi-group tests of profile similarity proposed by Morin, Meyer, Creusier and Biétry (2016) for latent profile analysis (LPA) with continuous indicators:

  1. Class enumeration in each group (how many profiles?);
  2. Configural similarity (same number of profiles in every group);
  3. Structural similarity (same profile means);
  4. Dispersion similarity (same within-profile variances);
  5. Distributional similarity (same profile proportions);
  6. Predictive similarity (same relations between predictors and profile membership);
  7. Explanatory similarity (same profile-specific levels of distal outcomes).

The models are diagonal Gaussian mixtures estimated by maximum likelihood with an EM algorithm and Mplus-style random starts. The package reproduces the log-likelihoods, free-parameter counts, information criteria and entropy that Mplus reports for the same models (the conventions are described at the end).

The random starts in this vignette are kept small so that it builds quickly; use many more in practice (for example starts = c(500, 50) or more). The EM iterations of the enumeration and similarity models run in compiled C++ code, and cores = n distributes the random starts over several CPU cores.

Data

The example data set contains five standardized working-memory indicators for 500 younger and 500 older adults, two predictors and two distal outcomes.

library(mglpa)
data(mglpa_wm)
head(mglpa_wm)
#>   id group verbal_wm visual_wm updating inhibition proc_speed education
#> 1  1     1     0.823     1.062   -0.441     -0.006     -0.127        14
#> 2  2     1    -0.990    -0.939   -1.988     -1.056      0.009        16
#> 3  3     1     0.197    -0.142    0.246     -0.724      0.312        18
#> 4  4     1    -0.627    -1.015   -1.292     -0.960     -1.216        19
#> 5  5     1    -1.644    -2.418   -1.090     -0.343     -0.709        17
#> 6  6     1    -1.095    -1.135   -1.477     -1.265     -0.409        16
#>   cog_reserve task_perf everyday_func
#> 1      -1.294     50.04         73.02
#> 2       0.041     37.94         71.35
#> 3       0.447     56.30         66.82
#> 4      -1.014     29.86         77.69
#> 5       0.190     28.26         64.68
#> 6      -0.232     28.78         56.13
ind <- c("verbal_wm", "visual_wm", "updating", "inhibition", "proc_speed")

Step 0: class enumeration

enum <- enumerate_profiles(mglpa_wm, ind, group = "group", k = 1:4,
                           starts = c(20, 4), seed = 1, verbose = FALSE)
#> Warning: mglpa (4-profile LPA; class-invariant variances):
#>   THE BEST LOG-LIKELIHOOD VALUE WAS NOT REPLICATED. The solution may not be trustworthy due to local maxima. Increase the number of random starts.
enum
#> <mglpa_enumeration> 4 solutions (k = 1-4) in 2 group(s); class-invariant variances
#> 
#> Group 1
#>  k        LL npar      AIC      BIC     CAIC     ABIC entropy VLMR_p aLMR_p
#>  1 -3248.899   10 6517.798 6559.944 6569.944 6528.203      NA     NA     NA
#>  2 -2571.691   16 5175.383 5242.817 5258.817 5192.032   0.936  0.000  0.000
#>  3 -2178.387   22 4400.773 4493.495 4515.495 4423.665   0.974  0.000  0.000
#>  4 -2170.418   28 4396.835 4514.844 4542.844 4425.971   0.933  0.218  0.229
#>  min_prop reps best conv
#>     1.000  4/4       yes
#>     0.376  4/4       yes
#>     0.208  4/4  *CA  yes
#>     0.046  1/4       yes
#> 
#> Group 2
#>  k        LL npar      AIC      BIC     CAIC     ABIC entropy VLMR_p aLMR_p
#>  1 -3207.710   10 6435.420 6477.566 6487.566 6445.825      NA     NA     NA
#>  2 -2643.239   16 5318.479 5385.913 5401.913 5335.128   0.892  0.000   0.00
#>  3 -2260.810   22 4565.620 4658.342 4680.342 4588.512   0.963  0.000   0.00
#>  4 -2251.311   28 4558.622 4676.631 4704.631 4587.757   0.892  0.046   0.05
#>  min_prop reps best conv
#>     1.000  4/4       yes
#>     0.476  4/4       yes
#>     0.148  4/4   *C  yes
#>     0.052  4/4    A  yes
#> 
#> * lowest BIC, C lowest CAIC, A lowest ABIC within group. VLMR_p / aLMR_p: Vuong-Lo-Mendell-Rubin tests of k versus k - 1 profiles (Mplus TECH11).  
#>   Inspect the elbow plot and the substantive meaning of the profiles before retaining k.
#> 
#> Estimation warnings:
#>   Group 1, k = 4: THE BEST LOG-LIKELIHOOD VALUE WAS NOT REPLICATED. The solution may not be trustworthy due to local maxima. Increase the number of random starts.
elbow_plot(enum)

The columns VLMR_p and aLMR_p are the p-values of the Vuong-Lo-Mendell-Rubin likelihood ratio test and of its adjusted version, which compare each solution with the solution with one profile less (the Mplus TECH11 output; see vlmr_test()). The bootstrap likelihood ratio test (TECH14) is added with enumeration_blrt(); very few draws are used here to keep the vignette fast:

enum <- enumeration_blrt(enum, B = 10, starts = c(5, 2), seed = 11, verbose = FALSE)
#> Warning: mglpa BLRT (2 vs 1 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 8 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.8000). Increase the number of random starts of the k-profile model (`starts`).
#> Warning: mglpa BLRT (3 vs 2 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 6 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.6000). Increase the number of random starts of the k-profile model (`starts`).
#> Warning: mglpa BLRT (4 vs 3 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 6 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.7000). Increase the number of random starts of the k-profile model (`starts`).
#> Warning: mglpa BLRT (2 vs 1 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 8 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.8000). Increase the number of random starts of the k-profile model (`starts`).
#> Warning: mglpa BLRT (3 vs 2 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 6 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.6000). Increase the number of random starts of the k-profile model (`starts`).
#> Warning: mglpa BLRT (4 vs 3 profiles):
#>   OF THE 10 BOOTSTRAP DRAWS, 5 HAD BOTH A SMALLER LRT VALUE THAN THE OBSERVED LRT VALUE AND NOT A REPLICATED (OR NOT CONVERGED) BEST LOG-LIKELIHOOD VALUE. THE P-VALUE MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA (it could be as large as 0.5000). Increase the number of random starts of the k-profile model (`starts`).
enumeration_table(enum)[, c("group", "k", "LRT", "VLMR_p", "aLMR_p", "BLRT_p")]
#>   group k        LRT       VLMR_p       aLMR_p BLRT_p
#> 1     1 1         NA           NA           NA     NA
#> 2     1 2 1354.41487 3.472223e-13 3.301193e-12    0.0
#> 3     1 3  786.60967 5.779544e-12 0.000000e+00    0.0
#> 4     1 4   15.93804 2.179133e-01 2.292043e-01    0.1
#> 5     2 1         NA           NA           NA     NA
#> 6     2 2 1128.94074 0.000000e+00 0.000000e+00    0.0
#> 7     2 3  764.85865 4.890977e-12 1.723066e-13    0.0
#> 8     2 4   18.99821 4.596091e-02 5.039955e-02    0.0

The classification quality of a solution (entropy and the average posterior probabilities of the observations assigned to each profile, the tables printed by Mplus) is obtained with classification_quality():

classification_quality(enum$fits[["1"]][["3"]])
#> Classification quality
#>   Entropy = 0.974
#> 
#> Average posterior probabilities for most likely profile membership (row) by profile (column):
#>             n Profile 1 Profile 2 Profile 3
#> Profile 1 104     0.988     0.012     0.000
#> Profile 2 218     0.008     0.990     0.003
#> Profile 3 178     0.000     0.008     0.992
#> 
#> Probabilities of the most likely profile (column) given the profile (row):
#>           Profile 1 Profile 2 Profile 3
#> Profile 1     0.984     0.016     0.000
#> Profile 2     0.006     0.988     0.006
#> Profile 3     0.000     0.003     0.997

The three-profile solution has the lowest BIC and CAIC in both groups. Its profiles can be inspected with profile_plot() (z scores computed with the sample mean and standard deviation of each indicator):

profile_plot(enum$fits[["1"]][["3"]], title = "Younger adults, 3 profiles")

Steps 1 to 4: similarity tests

sim <- similarity_tests(mglpa_wm, ind, group = "group", k = 3,
                        starts = c(30, 5), seed = 2, verbose = FALSE)
sim
#> <mglpa_similarity> 3-profile solution, 4 model(s), class-invariant variances
#>           model        LL npar      AIC      BIC     CAIC     ABIC entropy
#>      configural -5132.344   45 10354.69 10575.54 10620.54 10432.61   0.981
#>      structural -5138.613   30 10337.23 10484.46 10514.46 10389.18   0.981
#>      dispersion -5142.560   25 10335.12 10457.81 10482.81 10378.41   0.981
#>  distributional -5186.364   23 10418.73 10531.61 10554.61 10458.56   0.981
#>  decrease  decision reps conv
#>                      5/5  yes
#>       3/3 supported  5/5  yes
#>       3/3 supported  5/5  yes
#>       0/3  rejected  5/5  yes
#> 
#> Decision rule: at least 2 of CAIC, BIC and ABIC lower than the previous model.
#> Most constrained model supported: dispersion
#> 
#> Profile proportions by group (dispersion model):
#>      c1    c2    c3
#> 1 0.209 0.437 0.354
#> 2 0.447 0.406 0.147

Each model is compared with the previous one. Similarity is supported when at least two of CAIC, BIC and ABIC decrease. Here structural and dispersion similarity are supported and distributional similarity is rejected: the profiles have the same shape and the same within-profile variability in both groups, but their sizes differ.

profile_proportions(sim$fits$dispersion)
#>          c1        c2        c3
#> 1 0.2086981 0.4370536 0.3542483
#> 2 0.4472154 0.4057118 0.1470728
profile_plot(sim$fits$dispersion, type = "line", title = "Dispersion similarity model")

APA-style tables can be produced and written to a Word-compatible file:

apa_table(sim)
#>                       Model        LL fp       AIC       BIC      CAIC
#> 1     Configural similarity -5132.344 45 10354.688 10575.537 10620.537
#> 2     Structural similarity -5138.613 30 10337.226 10484.458 10514.458
#> 3     Dispersion similarity -5142.560 25 10335.119 10457.813 10482.813
#> 4 Distributional similarity -5186.364 23 10418.728 10531.607 10554.607
#>        ABIC Entropy Delta CAIC Delta BIC Delta ABIC  Decision
#> 1 10432.614   0.981                                          
#> 2 10389.177   0.981   -106.078   -91.078    -43.438 supported
#> 3 10378.412   0.981    -31.645   -26.645    -10.765 supported
#> 4 10458.557   0.981     71.794    73.794     80.146  rejected
# write_table_doc(apa_table(sim), "similarity.doc", title = "Table 2")

Step 5: predictive similarity

The profile solution is held fixed at the estimates of the most constrained model supported (the dispersion model), and profile membership is regressed on the predictors with group-specific or equal effects.

pred <- predictive_similarity(sim, predictors = c("education", "cog_reserve"),
                              seed = 3, verbose = FALSE)
pred
#> <mglpa_predictive> predictors: education, cog_reserve; profile solution fixed at the dispersion model
#>          model        LL npar      AIC      BIC     CAIC     ABIC  decision
#>   free_effects -8872.408   18 17780.81 17869.15 17887.15 17811.99          
#>  equal_effects -8872.914   14 17773.83 17842.54 17856.54 17798.07 supported
#> 
#> Predictive similarity supported (3 of 3 information criteria lower for equal effects).
#> 
#> free effects (logit coefficients and odds ratios):
#>  group_level   predictor      comparison   coef    se     p    OR
#>            1   education Profile 1 vs. 2  0.068 0.057 0.228 1.071
#>            1   education Profile 1 vs. 3  0.062 0.057 0.275 1.064
#>            1   education Profile 2 vs. 3 -0.006 0.044 0.884 0.994
#>            1 cog_reserve Profile 1 vs. 2 -0.094 0.145 0.516 0.910
#>            1 cog_reserve Profile 1 vs. 3 -0.217 0.152 0.152 0.805
#>            1 cog_reserve Profile 2 vs. 3 -0.123 0.122 0.313 0.884
#>            2   education Profile 1 vs. 2  0.035 0.035 0.324 1.035
#>            2   education Profile 1 vs. 3  0.051 0.044 0.252 1.052
#>            2   education Profile 2 vs. 3  0.016 0.045 0.721 1.016
#>            2 cog_reserve Profile 1 vs. 2  0.040 0.106 0.706 1.041
#>            2 cog_reserve Profile 1 vs. 3 -0.062 0.144 0.667 0.940
#>            2 cog_reserve Profile 2 vs. 3 -0.102 0.148 0.491 0.903
#> 
#> equal effects (logit coefficients and odds ratios):
#>  group_level   predictor      comparison   coef    se     p    OR
#>          all   education Profile 1 vs. 2  0.045 0.030 0.132 1.046
#>          all   education Profile 1 vs. 3  0.050 0.034 0.136 1.051
#>          all   education Profile 2 vs. 3  0.006 0.031 0.859 1.006
#>          all cog_reserve Profile 1 vs. 2 -0.003 0.085 0.975 0.997
#>          all cog_reserve Profile 1 vs. 3 -0.126 0.101 0.212 0.882
#>          all cog_reserve Profile 2 vs. 3 -0.123 0.094 0.190 0.884

Step 6: explanatory similarity

Distal outcomes are added as indicators whose means may differ across profiles and groups; the profile solution is again held fixed.

expl <- explanatory_similarity(sim, outcomes = c("task_perf", "everyday_func"),
                               outcome_variances = "equal", seed = 4, verbose = FALSE)
expl
#> <mglpa_explanatory> outcomes: task_perf, everyday_func; profile solution fixed at the dispersion model
#>                  model        LL npar      AIC      BIC     CAIC     ABIC
#>   free_means_equal_var -12675.59   14 25379.19 25447.90 25461.90 25403.43
#>  equal_means_equal_var -12742.17    8 25500.35 25539.61 25547.61 25514.20
#>  decision
#>          
#>  rejected
#> 
#> Explanatory similarity with equal outcome variances: rejected (0 of 3 criteria lower for equal means).
#> 
#> Outcome means by profile (free_means_equal_var):
#>  group_level class      variable   mean ci_lower ci_upper proportion
#>            1     1     task_perf 36.609   34.812   38.405  0.2086978
#>            1     1 everyday_func 66.354   64.365   68.343  0.2086978
#>            1     2     task_perf 51.318   50.204   52.432  0.4370542
#>            1     2 everyday_func 75.392   74.089   76.696  0.4370542
#>            1     3     task_perf 62.838   61.617   64.059  0.3542480
#>            1     3 everyday_func 84.703   83.149   86.258  0.3542480
#>            2     1     task_perf 34.218   32.784   35.653  0.4472139
#>            2     1 everyday_func 63.627   61.956   65.298  0.4472139
#>            2     2     task_perf 45.643   44.114   47.172  0.4057134
#>            2     2 everyday_func 72.002   70.244   73.761  0.4057134
#>            2     3     task_perf 55.134   52.788   57.480  0.1470727
#>            2     3 everyday_func 73.452   70.878   76.026  0.1470727
head(expl$contrasts$free_means_equal_var)
#>     outcome             type group_level profile            comparison
#> 1 task_perf between profiles           1       1 Profile 1 - Profile 2
#> 2 task_perf between profiles           1       1 Profile 1 - Profile 3
#> 3 task_perf between profiles           1       2 Profile 2 - Profile 3
#> 4 task_perf between profiles           2       1 Profile 1 - Profile 2
#> 5 task_perf between profiles           2       1 Profile 1 - Profile 3
#> 6 task_perf between profiles           2       2 Profile 2 - Profile 3
#>    estimate        se          z             p  ci_lower   ci_upper
#> 1 -14.70955 1.0784440 -13.639609  2.328056e-42 -16.82327 -12.595843
#> 2 -26.22914 1.1084970 -23.661891 8.904575e-124 -28.40175 -24.056521
#> 3 -11.51958 0.8432024 -13.661702  1.719175e-42 -13.17223  -9.866934
#> 4 -11.42448 1.0827144 -10.551698  4.988712e-26 -13.54656  -9.302395
#> 5 -20.91549 1.4018991 -14.919395  2.464980e-50 -23.66316 -18.167814
#> 6  -9.49101 1.4332058  -6.622224  3.538344e-11 -12.30004  -6.681978

Fitting a single model

Every model of the sequence can be specified directly:

m <- mglpa_model(3, ind, group = "group", equal_means = TRUE)
fit <- fit_lpa(mglpa_wm, m, starts = c(20, 4), seed = 5)
fit
#> <mglpa_fit> 3-profile LPA; class-invariant variances; equal across groups: means
#>   N = 1000 (1: 500, 2: 500)
#>   Log-likelihood = -5138.613, free parameters = 30 (Mplus KNOWNCLASS convention)
#>   AIC = 10337.226, BIC = 10484.458, CAIC = 10514.458, ABIC = 10389.177
#>   Entropy = 0.981
#>   Best log-likelihood replicated 4 time(s) in 4 final-stage optimisation(s)
#>   Profile proportions (model-based):
#>      c1    c2    c3
#> 1 0.354 0.437 0.209
#> 2 0.147 0.406 0.447
head(parameters(fit))
#>   type   variable class group group_level           label free         est
#> 1 mean  verbal_wm     1     1           1  m_verbal_wm_c1 TRUE 1.147474986
#> 2 mean  visual_wm     1     1           1  m_visual_wm_c1 TRUE 1.060708571
#> 3 mean   updating     1     1           1   m_updating_c1 TRUE 1.006771451
#> 4 mean inhibition     1     1           1 m_inhibition_c1 TRUE 0.890089395
#> 5 mean proc_speed     1     1           1 m_proc_speed_c1 TRUE 1.150545891
#> 6 mean  verbal_wm     2     1           1  m_verbal_wm_c2 TRUE 0.002687907
#>           se          z             p    ci_lower   ci_upper
#> 1 0.02525877 45.4287797  0.000000e+00  1.09796871 1.19698126
#> 2 0.03044799 34.8367383 6.761066e-266  1.00103161 1.12038553
#> 3 0.02744440 36.6840408 1.312217e-294  0.95298142 1.06056148
#> 4 0.02996897 29.7003668 7.595316e-194  0.83135129 0.94882750
#> 5 0.02518925 45.6760669  0.000000e+00  1.10117587 1.19991592
#> 6 0.02412251  0.1114273  9.112775e-01 -0.04459135 0.04996717

reorder_profiles() relabels the profiles (by level, by size, or with an explicit permutation); fixed_from_fit() extracts estimates to hold them fixed in a later model; contrast_estimates() computes any linear combination of parameters with delta-method standard errors.

The Shiny application

mglpa_app() opens a graphical interface with the same workflow: data import, enumeration, similarity tests, profile plots, predictive and explanatory similarity, tables and project files.

Conventions shared with Mplus

  • ESTIMATOR = MLR: standard errors are sandwich estimates based on the observed information matrix (se = "robust").
  • For KNOWNCLASS models Mplus includes the known-group proportions in the log-likelihood and counts them as free parameters, and (up to version 8) computes entropy over the latent class patterns (profiles x groups). fit_lpa() follows these conventions when mplus_convention = TRUE (the default), so that the values are directly comparable with Mplus output. Mplus 9 instead reports a profile-specific entropy (“Classification Quality for C”), which corresponds to fit$entropy_profiles.
  • When predictors of profile membership are brought into the model, Mplus treats them as multivariate normal variables; their saturated means, variances and covariances are added to the log-likelihood and to the parameter count. Regression coefficients and all model comparisons are unaffected.
  • Predictive and explanatory models hold the profile solution fixed at the values of the retained similarity model rounded to five decimals, like Mplus start values (digits = 5).
  • TECH11: the Vuong-Lo-Mendell-Rubin test uses the weight matrix of Vuong
    1. with the robust (sandwich) covariance matrices of the two models in place of their inverse information matrices, as Mplus does (Vermunt, 2024), p-values computed with Imhof’s method, and the adjustment of Lo, Mendell and Rubin (2001), which divides the statistic by 1 + 1 / (d log N). vlmr_test() reproduces the statistic, mean, standard deviation and p-values of the Mplus output; variant = "vuong" gives Vuong’s original matrix (the implementation of LatentGOLD).
  • TECH14: the bootstrap p-value is the proportion of bootstrap statistics at least as large as the observed one; in every generated data set the (k - 1)-profile model is started from the parent estimates and the k-profile model uses random starts (LRTSTARTS). Mplus chooses the number of draws adaptively, blrt() uses the fixed number B. Like Mplus, blrt() warns when draws with a statistic below the observed one did not replicate their best log-likelihood or did not converge (possible local maxima that understate the bootstrap statistic) and reports the largest p-value they could produce (p_upper); draws above the observed statistic whose (k - 1)-profile model did not converge may overstate it, and p_lower is the smallest p-value they could produce. Non-converged draws are always listed in the warnings.
  • Classification quality: the same two tables as Mplus (average posterior probabilities by most likely profile, and probabilities of the most likely profile given the profile); for KNOWNCLASS models they are computed within each known group.

References

Lo, Y., Mendell, N. R., & Rubin, D. B. (2001). Testing the number of components in a normal mixture. Biometrika, 88(3), 767-778.

Morin, A. J. S., Meyer, J. P., Creusier, J., & Biétry, F. (2016). Multiple-group analysis of similarity in latent profile solutions. Organizational Research Methods, 19(2), 231-254. https://doi.org/10.1177/1094428115621148

Vermunt, J. K. (2024). The Vuong-Lo-Mendell-Rubin test for latent class and latent profile analysis: A note on the different implementations in Mplus and LatentGOLD. Methodology, 20(1), Article e12467. https://doi.org/10.5964/meth.12467

Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 57(2), 307-333.