Research Online

Logo

Goldsmiths - University of London

Reconsidering cluster bias in multilevel data: A Monte Carlo comparison of free and constrained baseline approaches

Guenole, Nigel. 2018. Reconsidering cluster bias in multilevel data: A Monte Carlo comparison of free and constrained baseline approaches. Frontiers in Psychology, 9(255), ISSN 1664-1078 [Article]

[img] Text
270417_GuenoleFrontiers.pdf - Accepted Version
Permissions: Administrator Access Only
Available under License Creative Commons Attribution Non-commercial.

Download (652kB)
[img]
Preview
Text
fpsyg-09-00255.pdf - Published Version
Available under License Creative Commons Attribution.

Download (599kB) | Preview

Abstract or Description

The test for item level cluster bias examines the improvement in model fit that results from freeing an item’s between level residual variance from a baseline model with equal within and between level factor loadings and between level residual variances fixed at zero. A potential problem is that this approach may include a misspecified unrestricted model if any non-invariance is present, but the log-likelihood difference test requires that the unrestricted model is correctly specified. A free baseline approach where the unrestricted model includes only the restrictions needed for model identification should lead to better decision accuracy, but no studies have examined this yet. We ran a Monte Carlo study to investigate this issue. When the referent item is unbiased, compared to the free baseline approach, the constrained baseline approach led to similar true positive (power) rates but much higher false positive (Type I error) rates. The free baseline approach should be preferred when the referent indicator is unbiased. When the referent assumption is violated, the false positive rate was unacceptably high for both free and constrained baseline approaches, and the true positive rate was poor regardless of whether the free or constrained baseline approach was used. Neither the free or constrained baseline approach can be recommended when the referent indicator is biased. We recommend paying close attention to ensuring the referent indicator is unbiased in tests of cluster bias. All Mplus input and output files, R, and short Python scripts used to execute this simulation study are uploaded to an open access repository.

Item Type:

Article

Identification Number (DOI):

https://doi.org/10.3389/fpsyg.2018.00255

Related URLs:

Departments, Centres and Research Units:

Psychology

Dates:

DateEvent
15 February 2018Accepted
2 March 2018Published Online

Item ID:

22945

Date Deposited:

16 Feb 2018 09:42

Last Modified:

19 Aug 2019 14:42

Peer Reviewed:

Yes, this version has been peer-reviewed.

URI:

http://research.gold.ac.uk/id/eprint/22945

View statistics for this item...

Edit Record Edit Record (login required)