Author Topic: Characteristic Curves by Category  (Read 1427 times)

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Characteristic Curves by Category
« on: November 19, 2014, 10:39:27 AM »
Hi everyone!

I am working with MC-items and I want to analyse the characteristic curves "by category" to understand my items better. Plotting those curves, concquest (or poltquest) uses the raw data for the curves representing the probability for the distractors. Is there any possibility to get "smoothed" model porbability curves for all categories? Right now, conquest just shows this model probability curve for the right answer.

Thanks for you help!

Maria

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #1 on: November 20, 2014, 12:15:12 AM »
Hi Maria

You'd have to aplly a different model, the ordered partition model using the score statement:

score (a b c) (0 0 1);
The code a is scored as 0, b as 0 and c as 1 for all responses. As there are three separate codes in the codes list, the model that will be fitted if this score statement is used will have three response categories for each item. The actual model will be an ordered partition model because both the a and b codes have been assigned the same score.

Kind regards
Eveline

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #2 on: November 20, 2014, 02:09:33 PM »
Hi Eveline,

thanks a lot for your quick answer. I used the key-statement for the model (the items have 4 answer options 1, 2, 3, 4 and there is always one correct answer). The key statement in my syntax then indicates which answer is correct.
If there is no other possibility than using the ordered partition model, could somebody post the lines I would have to exchange?

Thanks!

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #3 on: November 21, 2014, 12:35:48 AM »
Dear maria

You need to use the score statement instead of the key statement.

Eveline

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #4 on: November 21, 2014, 12:39:32 AM »
In other words, skip the key statement and write

score (1,2,3,4)(1,0,0,0) !items(<list item numbers with 1 as the correct response>);
score (1,2,3,4)(0,1,0,0) !items(<list item numbers with 2 as the correct response>);
score (1,2,3,4)(0,0,1,0) !items(<list item numbers with 3 as the correct response>);
score (1,2,3,4)(0,0,0,1) !items(<list item numbers with 4 as the correct response>);

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #5 on: November 21, 2014, 12:17:27 PM »
Thanks again for the quick answer!

This is what my syntax looks like:

set constraints=cases;
datafile data_distr.dat;

format responses 9-31;

labels << labels_dist.lab;

score (1,2,3,4) (1,0,0,0) !item (16);
score (1,2,3,4) (0,1,0,0) !items (3,4,5,10,11,13,18,20,22);
score (1,2,3,4) (0,0,1,0) !items (1,2,6,12,14,19,23);
score (1,2,3,4) (0,0,0,1) !items (7,8,9,15,17,21);

codes 1,2,3,4;
model item;
estimate;

When conquest stops calculating, it says "Item thresholds are not available for models without discrete and increasing scores, within item multidimensional models, models with non-integer scores, or models with null categories".

When I look at the characteristic curves by category, I get a "smoothed" line for the correct response and only one "smoothed" line for the incorrect responses. But what I want is a "smoothed" line for every category.

What am I doing wrong?


Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #6 on: November 24, 2014, 03:02:31 AM »
Did you plot ICC (not MCC)?

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #7 on: November 24, 2014, 10:31:51 AM »
I plotted Characteristic Curves by Category (ICC).

May the model statement be the problem?

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #8 on: November 25, 2014, 01:11:04 AM »
Dear Maria

I am sorry, we think this may be a bug. It should work with your syntax. We will look into it later this week.

Best wishes
Eveline

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #9 on: November 30, 2014, 11:17:39 PM »
Dear Maria

You were right, you need a different model statement for the ordered partion model:

model item + item*step;

Best wishes
Eveline

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #10 on: December 15, 2014, 12:13:13 PM »
Hi all,

I have still the problem that Conquest says "Item thresholds are not available for models without discrete and increasing scores, within item multidimensional models, models with non-integer scores, or models with null categories".

And there is another thing I am wondering about: If I use the Rasch model with the key statement and compare it to the results I get from using the ordered partition model (see syntax above) I get different item difficulties. Why is that?


Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #11 on: December 15, 2014, 08:06:53 PM »
Hi Maria

The first issue is not a problem. It just means that thresholds cannot be produces for your model, but deltas can be produced.

I am not very familiar with the ordered partition model, but I will find out. In the mean time, you could have a look at this article:
http://bearcenter.berkeley.edu/sites/default/files/report%20-%20opm_parameters.pdf

Eveline

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #12 on: December 21, 2014, 10:56:09 PM »
Hi Maria

My colleague Greg is going to look into this model for you after he has recovered from the flu. He will reply to you using this forum.

Eveline

maria.bongartz

  • Newbie
  • *
  • Posts: 7
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #13 on: January 07, 2015, 10:55:56 AM »
Thank you!

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Characteristic Curves by Category
« Reply #14 on: March 12, 2015, 01:08:02 AM »
Hi Maria

I'm sorry my colleague hasn't been in touch with you.

Do you still have questions about this model?

Eveline