Characteristic Curves by Category

[maria.bongartz]

Hi everyone!

I am working with MC-items and I want to analyse the characteristic curves "by category" to understand my items better. Plotting those curves, concquest (or poltquest) uses the raw data for the curves representing the probability for the distractors. Is there any possibility to get "smoothed" model porbability curves for all categories? Right now, conquest just shows this model probability curve for the right answer.

Thanks for you help!

Maria

[Eveline Gebhardt]

Hi Maria

You'd have to aplly a different model, the ordered partition model using the score statement:

score (a b c) (0 0 1);
The code a is scored as 0, b as 0 and c as 1 for all responses. As there are three separate codes in the codes list, the model that will be fitted if this score statement is used will have three response categories for each item. The actual model will be an ordered partition model because both the a and b codes have been assigned the same score.

Kind regards
Eveline

[maria.bongartz]

Hi Eveline,

thanks a lot for your quick answer. I used the key-statement for the model (the items have 4 answer options 1, 2, 3, 4 and there is always one correct answer). The key statement in my syntax then indicates which answer is correct.
If there is no other possibility than using the ordered partition model, could somebody post the lines I would have to exchange?

Thanks!

[Eveline Gebhardt]

Dear maria

You need to use the score statement instead of the key statement.

Eveline

[Eveline Gebhardt]

In other words, skip the key statement and write

score (1,2,3,4)(1,0,0,0) !items(<list item numbers with 1 as the correct response>);
score (1,2,3,4)(0,1,0,0) !items(<list item numbers with 2 as the correct response>);
score (1,2,3,4)(0,0,1,0) !items(<list item numbers with 3 as the correct response>);
score (1,2,3,4)(0,0,0,1) !items(<list item numbers with 4 as the correct response>);

[maria.bongartz]

Thanks again for the quick answer!

This is what my syntax looks like:

set constraints=cases;
datafile data_distr.dat;

format responses 9-31;

labels << labels_dist.lab;

score (1,2,3,4) (1,0,0,0) !item (16);
score (1,2,3,4) (0,1,0,0) !items (3,4,5,10,11,13,18,20,22);
score (1,2,3,4) (0,0,1,0) !items (1,2,6,12,14,19,23);
score (1,2,3,4) (0,0,0,1) !items (7,8,9,15,17,21);

codes 1,2,3,4;
model item;
estimate;

When conquest stops calculating, it says "Item thresholds are not available for models without discrete and increasing scores, within item multidimensional models, models with non-integer scores, or models with null categories".

When I look at the characteristic curves by category, I get a "smoothed" line for the correct response and only one "smoothed" line for the incorrect responses. But what I want is a "smoothed" line for every category.

What am I doing wrong?

[Eveline Gebhardt]

Did you plot ICC (not MCC)?

[maria.bongartz]

I plotted Characteristic Curves by Category (ICC).

May the model statement be the problem?

[Eveline Gebhardt]

Dear Maria

I am sorry, we think this may be a bug. It should work with your syntax. We will look into it later this week.

Best wishes
Eveline

[Eveline Gebhardt]

Dear Maria

You were right, you need a different model statement for the ordered partion model:

model item + item*step;

Best wishes
Eveline

[maria.bongartz]

Hi all,

I have still the problem that Conquest says "Item thresholds are not available for models without discrete and increasing scores, within item multidimensional models, models with non-integer scores, or models with null categories".

And there is another thing I am wondering about: If I use the Rasch model with the key statement and compare it to the results I get from using the ordered partition model (see syntax above) I get different item difficulties. Why is that?

[Eveline Gebhardt]

Hi Maria

The first issue is not a problem. It just means that thresholds cannot be produces for your model, but deltas can be produced.

I am not very familiar with the ordered partition model, but I will find out. In the mean time, you could have a look at this article:
http://bearcenter.berkeley.edu/sites/default/files/report%20-%20opm_parameters.pdf

Eveline

[Eveline Gebhardt]

Hi Maria

My colleague Greg is going to look into this model for you after he has recovered from the flu. He will reply to you using this forum.

Eveline

[maria.bongartz]

Thank you!

[Eveline Gebhardt]

Hi Maria

I'm sorry my colleague hasn't been in touch with you.

Do you still have questions about this model?

Eveline