Examination of Parameter Estimation
Using Recursive Bayesian Analysis in Simulated Item Response Theory Applic
ations

\nby Robert C. Hendrick

For the past several years\, h igh-stakes testing has been the predominant indicator used to assess stude nts’ academic ability. School systems\, teachers\, parents\, and students are dependent upon the accuracy of academic ability estimates\, designated by es\, by item response theory (IRT) computer programs. In this study\, the accuracy of 3 parameter logistic (3PL) IRT estimates of academic abili ty were obtained from the BILOG-MG and WinBUGS computer programs which wer e employed to compare the use of non-informative and informative priors in e estimation. The rationale for comparing the output of these two compute r programs is that the underlying statistical theory employed in these two computer programs is different\, and there may be a notable difference in the accuracy of 6 estimation when an informative prior is used by Win BUG S in analyzing skewed populations. In particular\, the e parameter estimat es of BILOG-MG using traditionaiiRT analysis with non-informative priors i n each situation and the 6 parameter estimates of WinBUGS using Recursive Bayesian Analysis (RBA) with informative priors are compared to the true s imulated 6 value using Root Mean Square Errors (RMSEs). To make this compa rison\, Monte Carlo computer simulation is used across three occasions wit hin three conditions giving nine comparison situations. For the priors and data generated\, results show similar 6 estimation accuracy for a normall y distributed latent trait (RMSE = 0.35)\, a more accurate 6 estimation pr ocess using RBA compared to traditional analysis (RMSEs of 0.36 compared t o 0. 76) when using latent trait distributions skewed in a similar directi on\, and less accurate e estimation using RBA compared to traditional anal ysis (RMSEs of 1.48 compared to 0.80) when using extremely skewed negative then positive distributions in a longitudinal setting. Implications for f urther research include extensions to other IRT models\, developing prior elicitation equations\, and applying Bayesian informative prior elicitatio ns in BILOG-MG.

\n