### Author Topic: Fit indices in ConQuest  (Read 892 times)

#### navid

• Newbie
• Posts: 7
##### Fit indices in ConQuest
« on: November 29, 2013, 10:49:10 AM »
Hi,
We are doing a ConQuest analysis of a language proficiency test. We intend
to compare four models: a unidimensiona Rasch model, an MRCMLM with 6
factors and between-item multidimensionality, an MRCMLM with 5 factors and
between-item multidimensionality, and a testlet model (with one general
dimension and six specific factors). We want to compare the relative fits
of the models. I have a couple of questions and would be grateful to
receive an explanation from you.

1. Is the deviance statistic reported in ConQuest equal to the -2 Log
Likelihood statistic? Is it the same as G2 reported in Rasch papers?

2. How is the AIC and BIC statistics computed from deviance? (I know these can be estimated from chi-square but is an overall chi-square reported in ConQuest?)
3. Can we consider the testlet model to also have a hierarchical relation
to other models in our study? (so that we can compare their relative fit using the difference in their deviance indices).

Many thanks in advance.

Navid
« Last Edit: November 29, 2013, 10:51:11 AM by navid »

#### georgeha

• Newbie
• Posts: 5
##### Re: Fit indices in ConQuest
« Reply #1 on: December 02, 2013, 12:57:07 AM »
Hi Navid,

I'll give my best shot at answering your questions and see if anyone negates what I say.

1) Yes, the deviance statistic is the -2LL estimate (there's no need to take its log or multiply it by -2).
2) AIC = deviance + 2(n_parameters). BIC = deviance + Ln(N_persons)(n_parameters).
3) Good question.  Does anyone know?  It would make sense to me to consider the testlet model as nested since you're not modeling within-item multidimensionality. However, I'm not sure about comparing deviance statistics when you model inter-item dependence. I hope someone answers this.

George

#### navid

• Newbie
• Posts: 7
##### Re: Fit indices in ConQuest
« Reply #2 on: December 06, 2013, 03:41:19 PM »
Thank you George for your response.
As for the third question, we are fitting a testlet model which I assume falls within the category of within-item multidimensional models. Doesn't it? Can we still use the difference between deviances to compare relative fit?

Navid

#### Eveline Gebhardt

• Administrator
• Full Member
• Posts: 103
##### Re: Fit indices in ConQuest
« Reply #3 on: December 10, 2013, 09:16:44 PM »
1) Deviance  = -2 * loglikelihood. I don’t know what G2 is.

2) AIC  = 2* number of parameters – deviance
BIC = deviance + number of parameters * ln(number of cases)  ……  but I am not sure this is applicable

3) Yes, the testlet model is possible.  To decide if one model is a special case of another, you ask the question can I get from one to the other by setting some parameters to a fixed value or equal to each other.  Eg if you have two dimensional then set correlation=1 and you have one dimensional

#### Shirley

• Newbie
• Posts: 2
##### Re: Fit indices in ConQuest
« Reply #4 on: March 04, 2016, 08:14:52 AM »
If the number of cases differs across items (e.g., some participants skipped some items), is the total number of participants in the data set being used as number of cases in the calculation of BIC?

#### Eveline Gebhardt

• Administrator
• Full Member
• Posts: 103
##### Re: Fit indices in ConQuest
« Reply #5 on: March 18, 2016, 12:43:08 AM »
Hi Shirley

BIC is not computed by ConQuest but AIC is. The number of cases are not required for computing AIC.

Best wishes
Eveline

#### Isa89

• Newbie
• Posts: 6
##### Re: Fit indices in ConQuest
« Reply #6 on: October 23, 2020, 01:46:42 PM »
Hello everyone,

Is there a way to make ConQuest compute BIC and what do I need to write into my snytax then?

Best,
Isa

#### dan_c

• Administrator
• Jr. Member
• Posts: 89
##### Re: Fit indices in ConQuest
« Reply #7 on: November 09, 2020, 10:39:26 PM »
Isa,

BIC is reported in the output from the SHOW command in ConQuest Version 5. See example from Example 1 (https://www.acer.org/au/conquest/notes-tutorials):

Code: [Select]
`The Data File: ex1.datThe format:  id 1-5 responses 12-23No case weightsThe regression model:Grouping Variables:The item model: itemSlopes are fixedCases in file: 1000  Cases in estimation: 1000Final Deviance:                                13274.87615Akaike Information Criterion (AIC):            13300.87615Akaike Information Criterion Corrected (AICc): 13300.56785Bayesian Information Criterion (BIC):          13364.67697Total number of estimated parameters: 13The number of iterations: 45Termination criteria:  Max iterations=1000, Parameter Change= 0.00010                       Deviance Change= 0.00010`
You can download ConQuest 5 form the shop: https://shop.acer.edu.au/acer-conquest-5.html