Assessment: Question Selection [UPDATED]

Cirrus Assessment Support Team -

Step Three – Question Selection

When creating / editing Assessments, step three concerns the selection of questions to be available in your Assessment form. In the form itself you can choose to use them or not.

The previous steps and settings determine the options and questions available during Question selection and when assembling your assessment form.

Remember: depending on your choice during the initial setup of the Assessment, your will have an 

Manual question selection for an assessment: the 'Question Selection'-page

If the assessment has been set up for manual item selection, on the 'Question Selection'-tab you will see a list of all items available to you. 

2017-07-04_1635-assessment-question-selection.png

  1. Click the 'burger'-icon and use the filters in the left-hand panel to narrow down the list if needed. You can filter by: Collection, Marking Type, Taxonomy, Item Purpose, Difficulty, Question Type. So, for example, the assessment you are creating needs to have only Multiple Choice questions. Therefore you tick the Multiple Choice box in the Question Type filter so that only MC items are listed.
  2. Additionally you can filter on Labels by clicking the small filter icon. Searching for the labels you want to filter on, check the boxes for these labels and click on ‘Filter’. You can see if a filter is active because the icon is blacked.
  3. To select items for your assessment simply use the check-boxes and click on the 'Add Selected' button. Questions you've selected to be available for your Assessment form will have a 'checked'-icon - marked yellow in the screenshot above.
    Alternatively you can remove items from the selection of questions to be used in your Assessment forms by checking them and using the 'Remove'-button. No checked icon means they will not be available in the next step when building your Assessment form.
  4. You can see your selected score , a subtotal of points for your selection of questions and the amount of questions you've selected out of the total pool available to you. This helps you getting to the correct total score, for example 100 points and can help to check if you are working with the correct subset of questions.
  5. You can also search all listed Questions via the Search-box. Text entered will be looked up in both the question’s title and other fields. So you can for example also look up a question by it's question ID.
  6. Use the 'Share', Publish and [...] (View version history and preview Assessment) at your will.
  7. You can 'quick-preview any item before selecting it by clicking the magnifying glass symbol at the end of the row listing your question.

Which questions are available to me under 'Question Selection' / Help! I can't find my question to use in my assessment 

If you remember your training, you will know that the questions available to you will depend on:

  • Which collections you have access to.
  • Items need to be published (Live) before you can select them for an assessment
  • Which collection you are filtering on. You can easily filter out questions from a particular collection by clicking the 'burger'-icon and checking which collection to filter questions from.

Once you have selected all the items you need, go to 'Step 4: Generate assessment'.

 

 

Blueprint question selection for an assessment

When your assessment is setup using a blueprint in step one, you will need to use a matrix to have the system pull questions for you. On the 'Question Selection'-tab you:

  1. Select Taxonomies
  2. Select Subjects and Learning Objectives  
  3. [optional] Using this cog you can select which question types you wish to include if you only want specific question types in the assessment. Using this you can ensure only non-human-marked questions can be drawn so your assessment can be scored automatically.

2017-07-05_1127-assessment-question-selection-blueprint.png

Selecting the correct Taxonomies your Blueprint

As an assessment author you can use the dropdown to select the taxonomy repository containing the predefined terms to filter on. Read more about setting up taxonomy terms in Cirrus as an Admin and our article on using taxonomy terms in your e-assessments.

Selecting the correct Learning Objectives for your Blueprint

To start building your blueprint, you need to select one or more Learning Objectives that you wish to base the blueprint on.

Click the ‘Select Learning Objectives’ button. A pop-up will open allowing you to select the Learning Objective(s) you need.

  1. Select the repository from which to take the Learning Objectives
  2. Navigate to the right folder or Topic with Learning Objectives
  3. Check the boxes of the Learning Objectives to use
  4. Click ‘OK’ when you have finished and the pop-up will close.

 

 

Working on your blueprint: completing setting on your matrix to pull questions.

Once you have selected Learning Objectives and Taxonomies the blueprint will appear on your screen:

2017-07-05_1430-assessment-blueprint-example01.png

  1. Use the blueprint to select the minimum and maximum number of items you need of each Learning Objective and Subject against each taxonomy term.
    For example, there are a total of 24 items tagged with Learning Objective 'Bank Reconciliation' and taxonomy term 'Knowledge'. If I want 5 of these in my assessment I enter ‘5’ into both the Min and Max fields. I cannot set a minimum or maximum number for items tagged with Learning Objective 'Bank Reconciliation' and taxonomy term 'Understanding' as there are none available.
  2. To remove a Learning Objective or Subject use this button:
    2018-07-16_1151-Assessment-blueprint-remove-LO.png
  3. Click the ‘cog’ button in the top-right corner to open the ‘General Settings’ for the blueprint to define the Question types you wish to include if you only want specific question types in the assessment.
  4. Please note you can change the question selection matrix up until you generate an assessment form. After that you will not be able to change it and receive a warning stating "You have already created a form based on this blueprint. In order to change the blueprint you need to delete the form."
  5. Please note the remarks on which questions will be available in your matrix below:

Remember when using Blueprint:

      1. Available questions (1): When creating a blueprint assessment the questions available are depending on the type of assessment and will consist of only summative questions, consist of only formative questions, or consist of both types (mixed).
      2. Available questions (2): Items will/can be picked for a blueprint assessment if they are 'Live' and have a match on their Learning Objective and Taxonomy.
      3. Available questions (3): Only items with one Learning Objective will be available for selection using Blueprint.
      4. Question order: items will be presented in the order
        1. You add your Subjects to the blueprint
        2. Below that in the (alphanumeric) order of the Learning Objectives in your Blueprint
        3. To randomise the order select the option 'Display questions in random order' on your Assessment options
      5. The use of Learning Objectives in general and the feedback options it offers in Cirrus can be found via our article on 'Using Learning Objectives in your e-assessments'.
      6. Available questions (4): for (random) blueprint based assessments the candidates assessment will use questions that were Live when the schedule was created.

Example of available questions and their use over time in a random blueprint exam:

  1. I have a pool of 100 when the assessment is published. The assessment pulls 50 at random.
  2. I create 200 more questions (with LO's/Subjects and Taxonomies that match the blueprint)
  3. I schedule the assessment
  4. The scheduled assessment will pull 50 out of 300 questions at random.
  5. If you create your schedule a long time ahead of time you will only pull 50 out of 100 questions. In this case you need to create a new version of the assessment so your schedule-created-ahead-of-time will pick up the new questions (50 out of 300)
 

 

When you have finished setting up the blueprint, you are ready to generate the assessment. Once you have selected all the items you need, go to the next step ‘Step 4 – Generate Assessment’.

 

Comments 0

Cirrus uses Zendesk to provide our Help Center services. Check out Zendesk's Terms of Service and Privacy Policy, as they govern your use of these services.

Powered by Zendesk