Author Topic: Multidimensionality & reliability  (Read 809 times)

Georg II

  • Newbie
  • *
  • Posts: 21
    • View Profile
Multidimensionality & reliability
« on: January 26, 2014, 09:43:46 AM »
I have a scale with 25 items. I fitted a unidimensional model and a 5-dimensional model.
The unidimensional model fits significantly better. The correlations between dimensions are above .95 in the multidimensional model.
The strange thing is that EAP reliability of the unidimensional scale with 25 items is .82 while the EAP reliabilities of the subscales, each having only five items, are above .92. I could understand it if the multidimensional model had a better fit. Because then one could argue that forcing irrelevant items which belong to five different scales into a single dimension lowers reliability. But when the unidimensional model has a better fit why should it have lower reliability with 25 items than subscales which have only five items?
Isn’t this strange?

Georg II

« Last Edit: January 26, 2014, 09:51:37 AM by Georg II »

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Multidimensionality & reliability
« Reply #1 on: January 27, 2014, 11:02:25 PM »
Hello Georg II

Did both models converge?

Eveliine

Georg II

  • Newbie
  • *
  • Posts: 21
    • View Profile
Re: Multidimensionality & reliability
« Reply #2 on: January 28, 2014, 02:02:01 PM »
Hi Eveline,
I got this message when the 5-dimensioanl analysis converged:
"Convergence has occurred but a solution with a higher likelihood was encountered at iteration 148 The results for this earlier solution will be retained. Rerunning this analysis using current estimates as initial values and/or increasing the number of nodes is strongly advised".

The number of nodes was 1000. I reran the 5-dimensioanl analysis with 3000 nodes. It gives the same message at the end. This analysis has a better fit than the unidimensional analysis and the reliabilities are smaller (than the one with 1000 nodes) but still greater than the reliability of the single dimension with 25 items.
Do you think I should increase the number of nodes?


The show file for 1-dimensioanl model says "Iterations terminated because the convergence criteria were reached."
The other strange thing about this analysis is that this is the first time I see a unidimensional model fits better than a multidimensional model (when nodes=1000)!

Thanks
Georg II
« Last Edit: February 01, 2014, 10:00:39 AM by Georg II »

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Multidimensionality & reliability
« Reply #3 on: February 11, 2014, 09:46:59 PM »
Hi Georg

Both models need to converge for model comparison. It is not possible that a model with less parameters fits better. The question is if a model with more parameters fit significantly better or not.

Please have a look at the option keeplastest=yes for the set command. Looking in the log file or exporting a history file could also help in finding the problem of the 5 dim model. 3000 nodes should be enough. If this doesn't help, please send me all the input and output files and I'll have a look at it.

Eveline

Georg II

  • Newbie
  • *
  • Posts: 21
    • View Profile
Re: Multidimensionality & reliability
« Reply #4 on: February 12, 2014, 04:39:56 PM »
Thank you Eveline,
I tried the command "keeplastests=yes". It looks like:

model item + step;
Estimate ! nodes=3000, method=montecarlo, Keeplastest=yes;
show !estimate=latent >> run1.shw;
itanal >> run1.itn;

But it gives this error message: "Unknown command or argument: ‘keeplastest’"
Maybe I haven't written it in the right place?

Georg II

Eveline Gebhardt

  • Administrator
  • Full Member
  • *****
  • Posts: 103
    • View Profile
    • Email
Re: Multidimensionality & reliability
« Reply #5 on: February 12, 2014, 10:57:55 PM »
Yes, it is part of the set command. See also the command reference guide.

Eveline