Library: statistics on Items

Cirrus Assessment Support Team -

Library: statistics on Items

We sometimes get questions on the statistics shown in the Library for a Collection. This article briefly describes the values shown. 

Statistics for assessments will only be calculated if the results are published.

2019-08-15_Library-statistics_example.png

Example of the Statistics tab for a collection showing the question text, Item ID, Type of question and:

  • Chance score
  • Pc value - the corrected Difficulty value for the Item
  • P value - the difficulty value of the Item
  • Answered: how many times was the Item answered by a candidate
  • Given Answers: show the percentage of candidates that answered the question who choose a specific answer option
  • Status - offers an indicator on the difficulty of an item / 'OK' or a warning based on the P value of the item:
    • 'Warning': P smaller then 0.3 or higher then 0.7.
    • 'OK': P is in between 0.3 - 0.7

When evaluating your items you can also take into account the Rit value within the context of an assessment.

Chance Score

There is a difference between the chance and a chance score. Consider for example item I7281-55 in the screenshot above: this is a Multiple Choice question with 4 options. The chance in this case to pick the correct answer is 1 out of 4 or 25%). In this case the maximum score for the question is 1. This means that the chance score is 0.25 (25% from 1 points). If the question was worth 3 points, the chance score would be 25% of 3 = 0.75. For comparisons Cirrus always displays the Chance Score as a percentage of the maximum score for the question. [1]

P (difficulty)

What does it mean?
The P value is an indication if the question is hard (a lot of candidates answered it incorrectly) or easy (a lot of candidates answered it correctly). This might for example indicate it was too easy. It could also mean that all candidates understood what was expected of them or taught to them.

Calculation of the P-value:

P = average score of all candidates / max score for that question for all question types

Pc = Corrected Difficulty value

Also known as P' this is similar to the P-value but it takes the Chance score of the item into account.

Please note that Cirrus also offers statistics on assessments. For schedule statistics and assessment statistics different result may be shown because of aggregation of data.

Best Regards, the Cirrus Assessment Team

 

[1] For Fill in the Blank Hotspot, Sowiso, Numeric, Short answer, Essay and File response questions Chance score shows N/A. They are Not Available.

Comments 0

Cirrus uses Zendesk to provide our Help Center services. Check out Zendesk's Terms of Service and Privacy Policy, as they govern your use of these services.

Powered by Zendesk