Chi-Square value for MH statistics

[Chung Wei Kho]

Hi, why is it that when I run the mh command, sometimes there's an error saying "Chi-square value must be greater than 0"? For that particular item, the following is the output generated:
ConQuest: Generalised Item Response Modelling Software     Mon Jun 11 07:05 2018
Group: gender=1
DATA TABLE FOR EXPECTED SCORE
Estimator type is plausible value
===========================================================Build: Aug 18 2017===
item:29 (I29(40))
               Bins                       OBSERVED   EXPECTED
----------------------------------          GROUP     GROUP      CELL       CELL
     Low         High        Mean      N   AVERAGE   AVERAGE     CHISQ       P
--------------------------------------------------------------------------------
      <-      -0.30468    -1.28470   1271  0.439      0.476      7.130      0.008
  -0.30468     0.13248    -0.11180    106  0.934      0.746     19.729      0.000
   0.13248     0.56848     0.26860     54  1.000      0.811     12.554      0.000
   0.56848     1.12370     0.76018     27  1.000      0.876      3.839      0.050
   1.12370        ->       0.00000      0  0.000      0.767      0.000      1.000
================================================================================
ConQuest: Generalised Item Response Modelling Software     Mon Jun 11 07:05 2018
Group: gender=2
DATA TABLE FOR EXPECTED SCORE
Estimator type is plausible value
===========================================================Build: Aug 18 2017===
item:29 (I29(40))
               Bins                       OBSERVED   EXPECTED
----------------------------------          GROUP     GROUP      CELL       CELL
     Low         High        Mean      N   AVERAGE   AVERAGE     CHISQ       P
--------------------------------------------------------------------------------
      <-      -0.30468    -1.17305   1285  0.487      0.504      1.515      0.218
  -0.30468     0.13248    -0.11597    159  0.925      0.745     26.876      0.000
   0.13248     0.56848     0.29463     70  1.000      0.815     15.855      0.000
   0.56848     1.12370     0.70782     40  1.000      0.870      5.994      0.014
   1.12370        ->       0.00000      0  0.000      0.767      0.000      1.000
/* There are 2 scoring categories */
Mantel-Haenszel Statistics (between categories 0 and 1) is -1.#IO; CHISQ=1.#QO; df=4; p=2.00000
Suggested DIF Category is A: Negligible
The reference group is :1

[dan_c]

it may have something to do with your n=0 in your top bin ("1.12370        ->   ", for "Group: gender=1").

Can you upload your data so we can recreate it? or can you create a minimum reproducible example for us to recreate the error? Thats really the only way we can work out if it is a bug.

https://conquest-forums.acer.edu.au/index.php?topic=1239.0

[m.krell]

Hi,
I have another issue with the chi-square values for the MH statistics.

For one item, I have got the follwoing output:

...
Mantel-Haenszel Statistics (between categories 0 and 1) is -1.939; CHISQ=_BIG_; df=2; p=0.02520
Suggested DIF Category is C: Moderate to large
...

Could you give me further information about what is "BIG"? It would be very helpful for a publication, for example to report the value from which conquest doesn't provide the concrete value but big.

Thanks a lot!
Moritz

[dan_c]

Hi Moritz,

Can you run:

Code: [Select]

about;
at the console and let me know what version you are using? In more recent version of CQ, the actual value is reported.

[m.krell]

Hi Dan!

I tried your command and got the error message "unknown command or argument: 'about' ". But I know that I have conquest v.3 ...

The mentioned problem only occurs with this item. For all other items (probably with samller chi squares?) the value is reported.

Thanks! moritz

[dan_c]

Ah!

You can download version 4 from the website:

https://shop.acer.edu.au/acer-conquest-4

If you have purchased a Standard Licence, you can send the key (or your invoice) to the sales team and they will provide you with a new key to use with Version 4.

If the issue still persists, let me know.

[m.krell]

Hi Dan

Thanks for your support!

I downloaded version 4 and run the analysis again. But the value still isn't reported. Below you see what I get:

===========================================================
item:4 (BH_01)
               Bins                       OBSERVED   EXPECTED
----------------------------------          GROUP     GROUP      CELL       CELL
     Low         High        Mean      N   AVERAGE   AVERAGE     CHISQ       P
--------------------------------------------------------------------------------
      <-      -0.30407    -0.58653     33  0.091      0.341      9.158      0.002
  -0.30407     0.31765     0.07353     39  0.179      0.500     16.007      0.000
   0.31765        ->       0.89751     31  0.452      0.695      8.657      0.003
/* There are 2 scoring categories */
Mantel-Haenszel Statistics (between categories 0 and 1) is -1.939; CHISQ=_BIG_; df=2; p=0.02520
Suggested DIF Category is C: Moderate to large
The reference group is :0
=========================================================

[dan_c]

Hi all - this is a bug in the mh output of ConQuest. The output won't print CHISQ values > 10 (and instead will print "_BIG_"), but the value is being calculated and the associated output is correct.

Thank you to Moritz for providing a reproducible example to isolate it.

If this is an urgent issue for users, I can produce the correct output for you. We will push out a fix in the coming weeks.