Messages

1

Questions and Answers / Re: Read EAP file

« on: November 23, 2020, 08:46:05 PM »

The format of the EAP file is described in the manual:

https://conquestmanual.acer.org/s4-00.html#show

For plausible values (estimates=latent) and expected a-posteriori estimates (estimates=eap):

The file will contain one row for each case. Each row will contain (in order):

Sequence ID
PID (if PID is not specified in datafile or format than this is equal to the Sequence ID)
Plausible values. Note there will be np plausible values (default is 5) for each of nd dimensions. Dimensions cycle faster than plausible values, such that for nd = 2, and np = 3, the columns are in the order PV1_D1, PV1_D2, PV2_D1, PV2_D2, PV3_D1, PV3_D2.
the posterior mean (EAP), posterior standard deviation, and the reliability for the case, for each dimension. Note that these columns cycle faster than dimensions such that for nd = 2, and np = 3, the columns are in the order EAP_1, PosteriorSD_1, Reliability_1, EAP_2, PosteriorSD_2, Reliability_2.

If you use the option filetype to export a CSV, SPSS, or Excel file, you will see column headers providing a name for each of these columns.

2

Questions and Answers / Re: Fit indices in ConQuest

« on: November 09, 2020, 10:39:26 PM »

Isa,

BIC is reported in the output from the SHOW command in ConQuest Version 5. See example from Example 1 (https://www.acer.org/au/conquest/notes-tutorials):

Code: [Select]

The Data File: ex1.dat
The format:  id 1-5 responses 12-23
No case weights
The regression model:
Grouping Variables:
The item model: item
Slopes are fixed
Cases in file: 1000  Cases in estimation: 1000
Final Deviance:                                13274.87615
Akaike Information Criterion (AIC):            13300.87615
Akaike Information Criterion Corrected (AICc): 13300.56785
Bayesian Information Criterion (BIC):          13364.67697
Total number of estimated parameters: 13
The number of iterations: 45
Termination criteria:  Max iterations=1000, Parameter Change= 0.00010
                       Deviance Change= 0.00010
You can download ConQuest 5 form the shop: https://shop.acer.edu.au/acer-conquest-5.html

3

Questions and Answers / Re: Very different WLE Person separation reliability and EAP/PV reliability

« on: October 15, 2020, 05:52:40 AM »

Hi Isa,

If you have pre and post data, I would suggest estimating the growth in a single IRT model. One way is to create a wide file, load time 1 items onto one latent, and time 2 items onto a second latent, and anchor the difficulties of the items to be equal. The growth is then a function of the theta (theta2 - theta1 = growth). If you want to analyse the growth (e.g., regress covariates on the growth) you could choose a slightly different parameterisation, or you could then take PVs out of ConQuest and and fit whatever model (ANCOVA, mixed effects etc) you like in another stats package.

If you want to do it all in ConQuest, I would encourage you to read this article:

Wilson, M., Zheng, X., & McGuire, L. (2012). Formulating latent growth using an explanatory item response model approach. Journal of Applied Measurement, 13(1), 1.

I ran your model, and I think the difference between PV and WLE reliabilities is as described above. You have:

- Dim 1 and 2 have items that are very easy
- some cases have no observations on a dimension and therefore a WLE cannot be calculated
- generally few items per dimension (see Spearman-Brown prophecy formula to think about what the maximum reliability you might expect is)

4

Questions and Answers / Re: How to read a wle.csv file?

« on: October 13, 2020, 10:52:25 PM »

Hi Isa,

Check out the new, online ConQuest Manual. It includes lots of key information, including the formatting of output file:

https://conquestmanual.acer.org/s4-00.html#show

For maximum likelihood estimates and weighted likelihood estimates (estimates=mle or estimates=wle):

The file will contain one row for each case that provided a valid response to at least one of the items analysed (one item per dimension is required for multidimensional models). The row will contain the case number (the sequence number of the case in the data file being analysed), the raw score and maximum possible score on each dimension, followed by the maximum likelihood estimate and error variance for each dimension. The format is (i5, nd(2(f10.5, 1x)), nd(2(f10.5, 1x))). If the pfit option is set then an additional column is added containing the case fit statistics. The format is then (i5, nd(2(f10.5, 1x)), nd(2(f10.5, 1x)), f10.5)

5

Questions and Answers / Re: Very different WLE Person separation reliability and EAP/PV reliability

« on: October 13, 2020, 10:49:29 PM »

Hi Isa,

I can only see your labels file attached. Can you attach the rest of your files?

If you are trying to explain a population parameter, like average learning gain over time, I would recommend you use PVs. Point estimates, including EAPs will result in biased secondary analysis. See for example:

https://doi.org/10.1016/j.stueduc.2005.05.005

6

ConQuest News / Re: ACER ConQuest Manual and Command Reference

« on: July 06, 2020, 11:15:34 PM »

The ConQuest manual has a new home:

http://conquestmanual.acer.org

7

ConQuest News / Understanding Rasch Measurement Theory

« on: June 17, 2020, 07:34:31 AM »

The next iteration of Understanding Rasch Measurement Theory starts on Monday 13 July 2020.  This is a 100% online, 10 week, part-time, masters level course.
 
Led by world-renowned researcher and psychometrician Professor Geoff Masters, this masters level course will unlock the practical understanding you need to get the most from Rasch.
 
Find out more about enrolments, course outline and contents, and what to expect from an ACER online course below.
 
https://www.acer.org/au/professional-learning/postgraduate/rasch

8

Questions and Answers / Re: Very different WLE Person separation reliability and EAP/PV reliability

« on: May 26, 2020, 04:11:45 AM »

Thanks for the files.

I'd strongly encourage you to upgrade to a more recent version of ConQuest (version 5 is new this year).

Your model seems to run fine and converges happily. The estimated reliabilities are the same in a new build of ConQuest (attached).

The WLE reliability is low primarily because there are very few items in dimensions 2 and 3. Reviewing your WLE file, there are many cases where the max possible score for dimension 2 and 3 is 4.00. This can be interpreted as meaning there is little information available to space cases out along the latent trait. Your PV reliability is getting a boost because of the multidimenisfal structure. Your latent traits are highly correlated (> 0.8 ) and provide a lot of information that reduces the uncertainty in dimension 2 and 3. I think it's fair to conclude that this data is more useful for estimation of population parameters than individual abilities.

 

9

Questions and Answers / Re: Very different WLE Person separation reliability and EAP/PV reliability

« on: May 21, 2020, 07:31:51 AM »

we will need to see you data and outputs to check this. Can you also let us know what version of ConQuest you are running?

Alternatively, send through a reproducible example: https://conquest-forums.acer.edu.au/index.php?topic=1239.0

Note, you don't technically need your recode AND your score statements, see: https://conquestmanual.netlify.app/s4-00.html#score

Code: [Select]

codes 0,1,2,3;

score (0,1,2,3) (0,0,0.5,1) (   )   (   )   !items (1);
score (0,1)     (0,1)       (   )   (   )   !items (2,6,7,8,12,16,19,20,23,25,26,31,32,35,39,42,43,44,45,48);
score (0,1)     (0,1)       (   )   (   )   !items (4,17,18,34,38,46,49);
score (0,1,2)   (0,0.5,1)   (   )   (   )   !items (11,47);
score (0,1)     (0,1)       (   )   (   )   !items (10,21,24,33,37);
score (0,1,2)   (   )       (0,0,1) (   )   !items (5);
score (0,1)     (   )       (0,1)   (   )   !items (3,5,9,14,27,36,40);
score (0,1)     (   )       (   )   (0,1)   !items (13,15,22,28,29,30,41);
score (0,1,2)   (   )       (   )   (0,0,1) !items (30);


This is will give your items 1, 5, and 30, properties like in the ordered partition model.

10

Questions and Answers / Re: EAP/PV reliability estimates

« on: May 04, 2020, 06:21:37 AM »

If you are comparing "model 1 factor" to "model 4 factors", I think it makes sense to impose the same scoring regime in both models (0,1,2,3,4).

11

Questions and Answers / Re: EAP/PV reliability estimates

« on: April 30, 2020, 04:54:29 AM »

I'm not sure I follow - you would have to upload a minimum working example so I could recreate it.
https://conquest-forums.acer.edu.au/index.php?topic=1239.0

12

ConQuest News / ACER ConQuest Version 5: available now!

« on: April 30, 2020, 04:51:55 AM »

Version 5 of ACER ConQuest is now available.

Version 5 is a major upgrade on version 4 and includes new features including:

• Bayesian estimation using Markov Chain Monte Carlo (MCMC)
• Integration with R through the conquestr library

You can download ConQuest 5 here: https://shop.acer.edu.au/acer-conquest-5

13

Questions and Answers / Re: standard error of conditional covariances/correlations in multidimensional run?

« on: April 27, 2020, 01:17:50 AM »

you can extract the full error variance-covariance matrix and include in your Show file. See here:

https://conquestmanual.netlify.app/s4-00.html#show

You need to:

1. use empirical errors

Code: [Select]

estimate ! ... stderr = empirical ...;This is the default behaviour if you do not specify the stderr option.
2. add the option to your show command

Code: [Select]

show ! ... tables = 10 ...; Table 10 is the error variance-covariance matrix

14

Questions and Answers / Re: EAP/PV reliability estimates

« on: April 17, 2020, 12:27:03 PM »

You are presumably using default scoring, such that your “lowest” response category is scored 1 (the lowest value of your code statement). You need a zero-scored response category, and you do that using a score statement:

Code: [Select]

score (1,2,3,4,5) (0,1,2,3,4);

See: https://conquestmanual.netlify.app/s4-00.html#score

15

Questions and Answers / Re: set directory

« on: December 19, 2019, 07:35:38 AM »

Do you just copy and paste the desired directory in there?

yes, you can copy and paste a path into the `set` command.