Author Topic: Obtain Fit Statistics Like in Quest  (Read 135 times)

Juliane B.

  • Newbie
  • *
  • Posts: 4
    • View Profile
    • Email
Obtain Fit Statistics Like in Quest
« on: March 16, 2021, 10:07:27 AM »
Hey,

my colleagues and I are used to working with the old Acer Quest software. I have tried to switch to ConQuest, because I need ability estimates for some zero and perfect scores in my data set (Quest cannot calculate them). Unfortunately, none of us have found a way to receive the same fit statistics in ConQuest easily provided by Quest:
  • A summary of case estimates (mean and SD)
  • Overall fit statistics for the case estimates (Infit MNSQ and SD, Infit t Mean and SD)
  • Infit t values for all case estimates
Is it possible to obtain these values using ConQuest, and if yes, how?

Thanks a lot in advance!
Juliane

dan_c

  • Administrator
  • Jr. Member
  • *****
  • Posts: 99
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #1 on: March 16, 2021, 10:18:05 AM »
check out the output of the commands show and fit.

https://conquestmanual.acer.org/s4-00.html#show

Quote
tables =value list
If parameters output is requested, a total of eleven different tables can be produced.
...

The contents of the tables are:
...

2. The estimates, errors and fit statistics for each of the parameters in the item response model.
...
4.7.54.4 Examples

Code: [Select]
show;Produces displays with default settings and writes them to the output window.



https://conquestmanual.acer.org/s4-00.html#fit

Quote
Produces residual-based fit statistics.
...
4.7.25.4 Examples

Code: [Select]
fit >> fit.res;Uses the default fit design matrix and writes results to the file fit.res.



Juliane B.

  • Newbie
  • *
  • Posts: 4
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #2 on: April 09, 2021, 02:50:38 PM »
Dear Dan,

Thanks a lot for your fast reply!
The show command does provide me with all the parameter estimates I am looking for.
Unfortunately, I still cannot find the overall case estimates I need (a summary of case estimates - mean and SD; Overall fit statistics for the case estimates - Infit MNSQ and SD, Infit t Mean and SD). Is it possible to have these provided by conquest (as in quest) or do they need to be calculated by hand?
I am also wondering about the individual case fit values printed via pfit. Do these fit statistics represent the infit t value or do they represent the unweighted fit statistic?

Thank you very much.

Best,
Juliane

dan_c

  • Administrator
  • Jr. Member
  • *****
  • Posts: 99
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #3 on: April 12, 2021, 09:02:40 AM »
Hi Juliane,

in the show file you find the output from the population model which includes regression output and the latent variance-covariaince matrix (both unconditional, conditional and model R square if you include regressors in the population model). This is where you will find the mean and variance of the latent ability distribution. See e.g.:

Code: [Select]
REGRESSION COEFFICIENTS

Regression Variable

CONSTANT                   1.332 ( 0.115)
-----------------------------------------------
An asterisk next to a parameter estimate indicates that it is constrained
===============================================

UNCONDITIONAL COVARIANCE/CORRELATION MATRIX

Dimension

Dimension_1
-------------------------------------------
Variance                   0.625  ( 0.113)
-------------------------------------------
An asterisk next to a parameter estimate indicates that it is constrained

The person fit statistics in pfit are weighted mean squares/infit statistics with expectation 1.

Juliane B.

  • Newbie
  • *
  • Posts: 4
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #4 on: April 21, 2021, 08:27:54 AM »
Hi Dan,
thanks a lot! Right in front of me and I stilled missed it. That helps a lot!
Am I failing to see the person infit t-values (individuals and summary) as well? Is it possible to have them calculated by Conquest in addition to the weighted mean squares?
Best,
Juliane

dan_c

  • Administrator
  • Jr. Member
  • *****
  • Posts: 99
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #5 on: April 27, 2021, 11:17:55 AM »
Hi Juliane,

the output from the pfit does not include standard errors, and therefore also does not include t values.

It is relatively easy to add them in and I have put them on the work agenda for a future release of ConQuest.

Note that person fit statistics will generally be estimated relatively poorly (we tend to have fewer items per person, than vice-versa) and the t values may not be very useful. It may be more interesting to look at some visualisations - for example a qqplot of the quantiles of the pfits versus the quantiles of the std normal distribution. This will tell you if you have strong departure from the expectation.


Juliane B.

  • Newbie
  • *
  • Posts: 4
    • View Profile
    • Email
Re: Obtain Fit Statistics Like in Quest
« Reply #6 on: April 30, 2021, 01:03:04 PM »
Hi Dan,
I will look out for t-values in future updates then and check out the visual options you described now.
Thank you very much!
Best,
Juliane