Monte carlo Multidimensional models

[hanne72]

Hi
We have run 1-, 3- and 4-dimensional models using ConQuest 4. For the 3- and 4-dimensional models, we used the Monte Carlo estimation as suggested in the Tutorial. However, when applying the Monte Carlo approach for the 3-dimensional model we get deviance and AIC values that are about 25000 lower than without (the number of estimated parameters is equal for the two approaches). We also get much lower PSR for each of the dimensions and much higher regression coefficients (constants). For the 1-dimensional model, deviance was 69492, for the three-dimensional model without Monte Carlo 68816 (the four-dimensional model with Monte Carlo 69155). With Monte Carlo, deviance was 41956 for the three-dimensional model.  It seems unnatural that it could be that great differences in the results. What would you recommend in this case? Would you recommend using Monte Carlo estimation when analyzing the three-dimensional model?

[dan_c]

You would expect small differences between the residual deviances for models using gauss (default) and montecarlo integration - due to differences in precision.

The large difference you mention does seem far too big. Can you produce a small, reproducible example (perhaps using the example datasets https://www.acer.edu.au/conquest/notes-tutorials)? Or alternatively, perhaps you an email me your data and syntax?

[dan_c]

For info, I ran Example 7b (5 dimensions) with both default (Gauss - took a full week) and Monte Carlo estimation

Here's the output:

Estimation method was: Gauss-Hermite Quadrature with 759375 nodes
...
Cases in file: 583  Cases in estimation: 578
Final Deviance:     7948.35604
Akaike Information Criterion (AIC): 8090.35604
Total number of estimated parameters: 71
The number of iterations: 456

Estimation method was: MonteCarlo with 1000 nodes
...
Cases in file: 583  Cases in estimation: 578
Final Deviance:     7949.80685
Akaike Information Criterion (AIC): 8091.80685
Total number of estimated parameters: 71
The number of iterations: 258

This is the kind of difference in precision i would expect.

See:
https://www.acer.edu.au/files/Conquest-Tutorial-7-MultidimensionalModels.pdf

[hanne72]

Thank you very much for your answer! I have tried the same command as you and got approximately the same results (deviance 7951, AIC 8093 for both approaches). The command I have used is quite similar to what is stated in tutorial 7. I have tried to rerun the analyses using my command. The three-dimensional approach without Monte Carlo estimation runs without any problems. With Monte Carlo I got an error-message: ‘a dimension index is out of range for a regression initial value 4. Number of dimensions is 3’. I have use the following command:

title test_befolkning;
datafile befolkningCQ.dat;
format PID 1-6 responses 7-53;
codes 1,2,3,4;
score (1,2,3,4) (0,1,2,3) () () !items (1-16);
score (1,2,3,4) () (0,1,2,3) () !items (17-31);
score (1,2,3,4) () () (0,1,2,3) !items (32-47);
model item + item*step;
set warnings=no,update=yes;
export parameters >> befolkningCQ.prm;
export reg_coefficients >> befolkningCQ.reg;
export covariance >> befolkningCQ.cov;
import init_parameters << befolkningCQ.prm;
import init_reg_coefficients << befolkningCQ.reg;
import init_covariance << befolkningCQ.cov;
estimate!method=montecarlo,nodes=2000,converge=.005;
show cases !pfit=yes,estimates=wle,filetype=xls>> befolkningCQ _PersonFit.xls;
show !filetype=xls, estimates=mle,tables=1:2:3:4 >> befolkningCQ _ItemFit.xls;
itanal >> befolkningCQ _tradAnalyses.xls;
quit;

[Alvin Vista]

Try running it first without importing the init files.

[hanne72]

Thank you very much for your advice. When I ran without importing the infit files, the deviance and PSR became more similar to those from the analysis without Monte Carlo estimation.