Omega Hierarchical vs. Omega Total

Demonstrasi reliabilitas di konstruk multidimensional

Konteks

Slide Pertemuan 6 menunjukkan bahwa menggabungkan 5 item Extraversion (E1, E2, E3, E5, E8) dengan 5 item Neuroticism (N1, N2, N7, N8, N9) dari data IPIP Big Five (data/data.csv, n = 19.718) menghasilkan Cronbach’s \(\alpha = 0{.}616\). Koefisien ini terlihat tidak terlalu mengecewakan, padahal item-itemnya jelas mengukur dua konstruk yang berbeda.

Laman ini menunjukkan perhitungannya secara langsung di , termasuk kenapa McDonald’s ω di jamovi tidak menangkap masalah ini, sementara \(\omega\) hierarchical (dihitung lewat dekomposisi bifactor Schmid-Leiman di psych::omega()) dapat menangkapnya dengan jelas.

Setup

Code
library(psych) # install the package

data <- read.csv("../data/data.csv", sep = "\t") # import the dataset
data[data == 0] <- NA  # set 0 = missing 

rev_items <- c("E2", "E4", "E6", "E8", "E10", "N2", "N4") # tell R which items in Ext and Neu scale that are unfavorable
items <- c("E1", "E2", "E3", "E5", "E8", "N1", "N2", "N7", "N8", "N9") # we are only interested in these items. We combine these 5 items Ext (E1, E2, E3, E5, E8) and 5 items Neu (N1, N2, N7, N8, N9) into one single scale.

d <- reverse.code(ifelse(items %in% rev_items, -1, 1), data[, items], mini = 1, maxi = 5) # make a new dataset that contains only in the `items` vector, then reversing the score of unfavorable items

d <- na.omit(d) # get rid of participants with missing data for the sake of simplicity, we're essentially doing listwise deletion here

nrow(d) # how many participants do we have now after getting rid of those with missing data?
[1] 19718

Lihat mana item yang favorable dan unfavorable di kamus data: codebook.txt.

Cronbach’s alpha

Code
al <- alpha(d, check.keys = FALSE)$total$raw_alpha # calculate alpha -- you will see the alpha coefficient in the end of this code
Warning in alpha(d, check.keys = FALSE): Some items were negatively correlated with the first principal component and probably 
should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option
Some items ( E1 E2- E3 E5 E8- ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Omega (hierarchical dan total)

Code
om <- omega(d, nfactors = 2, fm = "minres", flip = FALSE, plot = FALSE,
            title = "E1,E2,E3,E5,E8 + N1,N2,N7,N8,N9") # now calculate omega -- here, you can see there than I specify `nfactors` = 2, because essentially we know that this scale consists of two latent constructs (Ext and Neu)
Loading required namespace: GPArotation

Three factors are required for identification -- general factor loadings set to be equal. 
Proceed with caution. 
Think about redoing the analysis with alternative values of the 'option' setting.
E1,E2,E3,E5,E8 + N1,N2,N7,N8,N9 
Call: omegah(m = m, nfactors = nfactors, fm = fm, key = key, flip = flip, 
    digits = digits, title = title, sl = sl, labels = labels, 
    plot = plot, n.obs = n.obs, rotate = rotate, Phi = Phi, option = option, 
    covar = covar)
Alpha:                 0.61 
G.6:                   0.74 
Omega Hierarchical:    0 
Omega H asymptotic:    0 
Omega Total            0.76 

Schmid Leiman Factor loadings greater than  0.2 
        g   F1*   F2*   h2   h2   u2   p2  com
E1  -0.29        0.59 0.43 0.43 0.57 0.19 1.46
E2- -0.27        0.61 0.45 0.45 0.55 0.16 1.42
E3  -0.40        0.62 0.56 0.56 0.44 0.28 1.87
E5  -0.35        0.68 0.59 0.59 0.41 0.21 1.49
E8- -0.21        0.45 0.25 0.25 0.75 0.17 1.43
N1   0.31  0.54       0.39 0.39 0.61 0.25 1.64
N2-  0.26  0.41       0.24 0.24 0.76 0.28 1.84
N7   0.34  0.71       0.62 0.62 0.38 0.19 1.46
N8   0.37  0.74       0.68 0.68 0.32 0.20 1.47
N9   0.33  0.59       0.46 0.46 0.54 0.23 1.57

With Sums of squares  of:
  g F1* F2*  h2 
1.0 1.9 1.8 2.4 

general/max  0.42   max/min =   1.33
mean percent general =  0.22    with sd =  0.04 and cv of  0.2 
Explained Common Variance of the general factor =  0.21 

The degrees of freedom are 26  and the fit is  0.3 
The number of observations was  19718  with Chi Square =  5944.95  with prob <  0
The root mean square of the residuals is  0.03 
The df corrected root mean square of the residuals is  0.04
RMSEA index =  0.107  and the 90 % confidence intervals are  0.105 0.11
BIC =  5687.83

Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 35  and the fit is  2.23 
The number of observations was  19718  with Chi Square =  43954.33  with prob <  0
The root mean square of the residuals is  0.17 
The df corrected root mean square of the residuals is  0.2 

RMSEA index =  0.252  and the 90 % confidence intervals are  0.25 0.254
BIC =  43608.21 

Measures of factor score adequacy             
                                                  g  F1*  F2*
Correlation of scores with factors             0.55 0.83 0.82
Multiple R square of scores with factors       0.30 0.69 0.67
Minimum correlation of factor score estimates -0.40 0.38 0.33

 Total, General and Subset omega for each subset
                                                 g  F1*  F2*
Omega total for total scores and subscales    0.76 0.81 0.80
Omega general for total scores and subscales  0.00 0.18 0.17
Omega group for total scores and subscales    0.79 0.63 0.63
WarningLihat error message di output

psych menampilkan peringatan “Three factors are required for identification – general factor loadings set to be equal. Proceed with caution” di bagian atas output. Dekomposisi bifactor idealnya butuh ≥3 dimensi/faktor untuk identifikasi yang stabil; dengan hanya 2 (Extraversion dan Neuroticism), \(\omega_h\) di atas adalah estimasi yang belum benar-benar stabil.

Ringkasan

Code
data.frame(
  Koefisien = c("Cronbach's alpha", "Omega hierarchical", "Omega total"),
  Nilai = round(c(al, om$omega_h, om$omega.tot), 3)
) # lets create a simple table consisting of alpha, omega hierarchical and omega total
Koefisien Nilai
Cronbach’s alpha 0.616
Omega hierarchical 0.000
Omega total 0.761

Interpretasi: \(\alpha\) dan G.6 tetap terlihat “cukup baik” karena keduanya hanya melihat rata-rata korelasi antar-item dan di dalam masing-masing subskala korelasinya memang tinggi.

Sedangkan \(\omega_h\) mendekati nol karena secara struktural tidak ada faktor umum tunggal yang mendasari kesepuluh item ini; yang ada adalah dua konstruk psikologis (Extraversion, Neuroticism) yang berkorelasi negatif kecil (tidak berhubungan) satu sama lain.

Kenapa jamovi tidak menunjukkan ini

jamovi menghitung McDonald’s \(\omega\) dari satu model faktor congeneric (bukan dekomposisi bifactor), dan menormalisasinya dengan varians total yang diobservasi, bukan varians yang diimplikasikan model.

Itulah kenapa koefisiennya tetap tinggi meski struktur datanya jelas dua dimensi. McDonald’s \(\omega\) tidak mampu “melihat” bahwa data Anda punya struktur multidimensional dengan dua dimensi ynang berbeda.

Untuk mendeteksi multidimensionalitas lewat \(\omega\), dekomposisi bifactor (psych::omega() di ) adalah cara yang tepat. Alternatifnya, Anda bisa menggunakan online calculator dari Lensym.

Reka-ulang analisis ini

Code
sessionInfo()
R version 4.6.1 (2026-06-24 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26200)

Matrix products: default
  LAPACK version 3.12.1

locale:
[1] LC_COLLATE=German_Germany.utf8  LC_CTYPE=German_Germany.utf8   
[3] LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C                   
[5] LC_TIME=German_Germany.utf8    

time zone: Europe/Berlin
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] psych_2.6.5

loaded via a namespace (and not attached):
 [1] digest_0.6.39        fastmap_1.2.0        xfun_0.60           
 [4] lattice_0.22-9       GPArotation_2026.8-2 knitr_1.51          
 [7] parallel_4.6.1       htmltools_0.5.9      rmarkdown_2.31      
[10] cli_3.6.6            grid_4.6.1           mnormt_2.1.2        
[13] compiler_4.6.1       tools_4.6.1          nlme_3.1-170        
[16] evaluate_1.0.5       yaml_2.3.12          otel_0.2.0          
[19] rlang_1.3.0          jsonlite_2.0.0       htmlwidgets_1.6.4