ConQuest Community
Help desk => Questions and Answers => Topic started by: navid on November 29, 2013, 10:49:10 AM

Hi,
We are doing a ConQuest analysis of a language proficiency test. We intend
to compare four models: a unidimensiona Rasch model, an MRCMLM with 6
factors and betweenitem multidimensionality, an MRCMLM with 5 factors and
betweenitem multidimensionality, and a testlet model (with one general
dimension and six specific factors). We want to compare the relative fits
of the models. I have a couple of questions and would be grateful to
receive an explanation from you.
1. Is the deviance statistic reported in ConQuest equal to the 2 Log
Likelihood statistic? Is it the same as G2 reported in Rasch papers?
2. How is the AIC and BIC statistics computed from deviance? (I know these can be estimated from chisquare but is an overall chisquare reported in ConQuest?)
3. Can we consider the testlet model to also have a hierarchical relation
to other models in our study? (so that we can compare their relative fit using the difference in their deviance indices).
Many thanks in advance.
Navid

Hi Navid,
I'll give my best shot at answering your questions and see if anyone negates what I say.
1) Yes, the deviance statistic is the 2LL estimate (there's no need to take its log or multiply it by 2).
2) AIC = deviance + 2(n_parameters). BIC = deviance + Ln(N_persons)(n_parameters).
3) Good question. Does anyone know? It would make sense to me to consider the testlet model as nested since you're not modeling withinitem multidimensionality. However, I'm not sure about comparing deviance statistics when you model interitem dependence. I hope someone answers this.
George

Thank you George for your response.
As for the third question, we are fitting a testlet model which I assume falls within the category of withinitem multidimensional models. Doesn't it? Can we still use the difference between deviances to compare relative fit?
Navid

1) Deviance = 2 * loglikelihood. I don’t know what G2 is.
2) AIC = 2* number of parameters – deviance
BIC = deviance + number of parameters * ln(number of cases) …… but I am not sure this is applicable
3) Yes, the testlet model is possible. To decide if one model is a special case of another, you ask the question can I get from one to the other by setting some parameters to a fixed value or equal to each other. Eg if you have two dimensional then set correlation=1 and you have one dimensional

If the number of cases differs across items (e.g., some participants skipped some items), is the total number of participants in the data set being used as number of cases in the calculation of BIC?

Hi Shirley
BIC is not computed by ConQuest but AIC is. The number of cases are not required for computing AIC.
Best wishes
Eveline

Hello everyone,
Is there a way to make ConQuest compute BIC and what do I need to write into my snytax then?
Best,
Isa

Isa,
BIC is reported in the output from the SHOW command in ConQuest Version 5. See example from Example 1 (https://www.acer.org/au/conquest/notestutorials (https://www.acer.org/au/conquest/notestutorials)):
The Data File: ex1.dat
The format: id 15 responses 1223
No case weights
The regression model:
Grouping Variables:
The item model: item
Slopes are fixed
Cases in file: 1000 Cases in estimation: 1000
Final Deviance: 13274.87615
Akaike Information Criterion (AIC): 13300.87615
Akaike Information Criterion Corrected (AICc): 13300.56785
Bayesian Information Criterion (BIC): 13364.67697
Total number of estimated parameters: 13
The number of iterations: 45
Termination criteria: Max iterations=1000, Parameter Change= 0.00010
Deviance Change= 0.00010
You can download ConQuest 5 form the shop: https://shop.acer.edu.au/acerconquest5.html (https://shop.acer.edu.au/acerconquest5.html)