A Simulation Study Using a Mixed Model Framework to Analyze the Impact of Sample Size and Variability on Type I Error and Power

Abstract

Repeated measures design (or longitudinal study) are commonly seen in many research fields, especially in pharmaceutical clinical trials, agricultural research, and psychology. PROC MIXED (SAS Inc.) is a well-known standard tool for analyzing repeated measures data nowadays. The MIXED procedure is based on the standard linear MIXED model, which estimates parameters by maximizing the restricted likelihood. The usual assumption for a standard linear MIXED model is normality. However, the character of data in the real world is hard to tell; it may be non- smoothed, non-symmetric, and having heavy tails, having a small sample size, and so on. Therefore, this simulation study was conducted to check the validity of a MIXED model's statistical inference when violating the underlying assumptions – normality of random errors [Scheffe, 1959], and giving two design features as unbalanced group size and inequality of variance of errors [Scheffe, 1959]. We compare the Type I error rate in different combinations of settings with the Type I error rate under the normal distribution. The power rate is also provided for checking the robustness. The main results in this study show us that the MIXED model is reasonably robust to modest violations of the normal distribution. In the meantime, when small group size combines with large variance, it would cause a severe inflation problem on Type I error rates, which breaks the MIXED model's performance. When the Type I errors were found to be inflated, the Group= option was found to often help with this problem, or sometimes one could use a Sub-Sampling procedure.