Library: the different Item types

Cirrus Assessment Support Team -

Introduction to Item Types and more

This article will list the different item or question types you can use in Cirrus.

  1. Working with question in Cirrus using Multiple Choice as an example
  2. Don't forget to check out the (advanced) options for questions.
  3. Using a Resource file (PDF) or using  a Media File (large image)
  4. Familiarise yourself with our additional item types like welcome and finish pages, sections, the Sowiso question.

Closed questions

Open questions

 marked automatically

 marked manually

 

  Additional item types

Working with questions in Cirrus using Multiple Choice as an example

You can create a Multiple Choice question in a collection in your Library by clicking the  ‘+ Add’ button, the question editor screen will open:

Example MC question - editor screen:

2018-12-06_1722-Library-example-MC-item-editing.png

  1. Add / Edit questions - this is where you create the question including the answer alternatives
    1. 'Enter the Question' - Click inside the top box to enter your question – you will see that a text editor appears when you do this, allowing you a range of text editing options (this is the same across all question types).
    2. 'Enter the alternatives': Next, enter your answer alternatives into the boxes available. By default, there will be 3 answer boxes – click on the ‘Add Alternative’ button to add more. Or, if you wish to delete any answer boxes, use the ‘Remove’ button to the right-hand side.
    3. To select the correct answer, simply click on the ‘Correct’ radio button next to the appropriate answer box
    4. Set maximum score for this question: next you will also have to set a maximum score for the question. Simply enter a number into the box. Some questions offer the option to choose between dichotomous and polytomous scoring - see the Multiple Response question for more information.
    5. You can use the ‘Preview’ button to see what the question will look like to the candidate and make sure that the correct answer receives the correct marks.
    6. Click the ‘Save’ button to save the item (or ‘Save and Close’ if you wish to return to your collection overview).
    7. Note that Latex code will also be properly rendered for most question fields using the WYSIWYG editor. Please note that this like the Sowiso question type will not be printable as only a limited amount of issue types are converted to print.
  2. Options button of a question - click the options button to access / setup options of the question. Multiple Choice and Multiple Response questions come with an additional options button for randomising the order of answer alternatives (default), fix the position of certain alternatives and enter feedback per alternative (the latter is also available for Either/Or questions). Read more about these options on alternatives.
    For all items except Essay and mathematical (Sowiso) questions you can also set feedback at the item level in case the question is answered correct/partially correct or wrong by the candidate. See "feedback on questions", fix positions of alternatives and even enter feedback per alternative.
  3. You can determine further properties or tags for the item and access to the revision history are using the right side-bar of any question:

    1. The first part ('tag' icon) shows the item's workflow status, the item purpose and any Learning Objectives and Taxonomy terms that apply. You can also view who it's been shared to work on.

    2. With and click on the clip-board icon you can see the Item Revision history (versions and accompanying comments by authors). Read all about this in our article 'Library: Adding an Item and Item settings'.

Item Options or advanced item options for your questions

Advanced Item Options: randomise order of options or fix position

Multiple Choice and Multiple Response questions come with an additional options to randomise the order of alternatives or to fix the position of certain alternatives:

 

2018-02-01_1535-Library-item-fix-answer-option-position.png

 

  • Randomise order - when checked means each candidate will see the alternatives in a different order when taking the exam. Leave unchecked to show the same order to each candidate in delivery.
  • Fixed position of alternatives - Allows you to fix positions of  certain alternatives, while using random ordering for the rest.

    2018-02-01_1535-Library-item-fix-answer-option-position-2.png

    Using this button you can choose to randomise the answer options but fix the position of one or more of them. For example if you want the bottom alternative to always show "All of the above"
  • "Feedback per alternative" -  using the "show feedback" option allows you to enter feedback on each alternative of your MC question (as opposed to feedback on the question as a whole).

Advanced Item Options: restriction on answer options to select for candidates

For Multiple Response and Extended Match question you can choose to limit the options (MR) connections (ExtMatch) a candidate can check:

2018-11-07_1213-Item-advanced-option-max-to-select.png

When combined with polytomous scoring this greatly changes the scoring for the question. See our article "Marking: dichotomous and polytomous scoring on items" for examples

Using a Resource file (PDF) or Media File (large image) with your questions or exam

Adding a Resource file (PDF)

Using the Resource button when editing a question or page in Cirrus allows the author to attach an PDF which will display in a separate modal window for the candidate when opening it. (Max size of the uploaded/added file is 10MB).

  • For the candidate a 'resource-button' shows at the bottom of the screen when taking the exam on the page you added the PDF to.
  • Clicking this opens a PDF in a separate window for the candidate
  • The candidate can have one PDF open at a time.
  • Option to search the PDF is available to the candidate.
  • The author can choose to leave the PDF open while navigating (the candidate can close it using the 'x'. Or to check the box 'close when navigating' so the PDF will close for the candidate when he/she moves to the next question.

2019-01-03_1709-example-with-PDF.png

If you add a PDF to a welcome page and choose to leave it open for the candidate it will close when the candidate chooses this or when they encounter a question with a resource attached.

Adding PDFs to sections in Cirrus will follow at a later date. If you have any questions or feedback don't hesitate contact us via support@cirrusassessment.com.

Please note: do not use password protected files / PDFs. The author will receive no error message when uploading these, but the candidate will not be able to open them when taking their exam!

Adding a Media file (large image)

When you add images to a question they are scaled down ensure uniform delivery to candidates. Using the media file button in a question you can add larger images that candidates can click it to enlarge it during the exam:

  • Allow opening in pop-up: The image is shown at a set position in the question, when opened in the pop-up by the candidate they are still scaled in order to make assessments pixel-perfect and presentation consistent:

2019-01-03_1806-media-file-large-image-example.png

 

  • The pop-up window with the enlarged image opens up in an overlay and can be moved - so the candidate can read the question for example
  • If the image is still larger than the pop-up the candidate can scroll
  • The candidate can close the pop-up using the 'X' as a close button.

Please note: do not confuse the use of the media file with adding images or video inline in your questions or answer alternatives. See:

 

Multiple Response

The Multiple Response (MR) question type is similar to the Multiple Choice question type, only it allows you to set more than one correct answer to be selected by the candidate.

  1. When the item editor screen opens, type your question into the question box.
  2. Enter each alternative answer options into the answer boxes.
    1. You can add and remove alternatives in the same way as with the multiple choice question type.
    2. Use the ‘Correct’ tick boxes to select the correct answers - only this time you will be able to select more than one if you wish.
  3. You will be able to randomise the order upon delivery to your candidates if you wish, using the option at the bottom of the screen.
  4. Set the maximum score for the question.
  5. Scoring type': this Item type uses Dichotomous scoring by default. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.
  6. he 'Show feedback' link opens up feedback boxes for you to enter feedback per alternative.
  7. Finally: never forget to click ‘Save’ when you have finished.
image-013.png

 

Extended Multiple Choice question

The  Extended MC question which combines the power of Multiple Response and Multiple Choice in one template:

[image]

 

Either/Or

Use the Either/Or question type for questions that only have two alternative answer options to choose from.

You create the question in the same way as a multiple choice question, but you will not be able to add or remove any alternative answer options – it will always be two alternatives. Also, it is only possible for one option to be correct. This question type is also sometimes known as yes/no or true/false questions. It offers the option for 'feedback per alternative or per question'. 

Numeric

The Numeric question type allows the candidate to enter a numeric response in order to answer the question. This can include alphanumeric characters as well if you wish, although you can set the question up so that they are ignored.

  1. Start by entering your question into the question box. Again, you will see a text editor appear when you do this and for this question type it may be useful to know that you can insert a table into the question box.
  2. Enter the correct answer. You have the option to enter either a specific value, or a range.
    Specific value The candidate has to enter the exact value to score

    2019-08-13_1208Lspecific_value.png

    Range Setting a range means that any number given between the minimum and maximum values stated will be marked as correct.

    2019-08-13_1229_range.png

  • ‘Include a working out box for candidate’ - this gives the candidate room to take notes / note down calculations.
  • Ignore non-alphanumeric characters in the candidate’s response - sets the computer-marking less strictly.
  • Decimal marker - you can set the decimal delimiter. other delimiters will be ignored: candidates simply cannot enter them when answering.

Note: Don't forget to preview your Item - this is a particularly useful way of checking that the marks have been assigned correctly for computer-marked questions. It will also give a first idea of what it looks like for the candidate.

Always remember to ‘Save’ the item before navigating away from this screen. 

Select from List (SFL)

The Select from List (SFL) question type requires the candidate to select their answer from a predefined list of possible correct answers. For example, the candidate may be required to select the correct word within a given sentence.

  1. Enter your question text into the box provided. For example:
    ‘A Multiple-Choice question always has multiple answering options. Considering this, fill in the blank in the next sentence:’
  2. Set up the answer: enter text into the answer box and then, blank out the word you want the candidate to provide (i.e. the correct answer) by highlighting it and clicking the ‘Create Blank’ button located at the bottom right-hand corner of the answer box.
    1. Once you have added the blank, the highlighted words will be replaced with a highlighted number in brackets. The highlighted words will appear under ‘Blanked words’ below the answer box.
    1. If needed, you can edit blanked word(s) by simply clicking on them and typing the correct answer.
    2. Enter any distractors into the ‘Distractors’ field, separating multiple alternatives with commas. The alternatives will appear in a dropdown list when viewed by the candidate during their assessment.
    3. To delete the blanked word(s), and any distractors, click on ‘Remove’ at the end of the row.
    4. To add another blank, highlight another word and click the ‘Create Blank’ button to repeat the process.
  1. Set the maximum score for the question. 'Scoring type': this Item type uses Dichotomous scoring by default. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.
  2. Don’t forget hit ‘Save’.

Fill in the Blanks (FITB)

The Fill in the Blanks question type requires the candidate to fill in a blank. This may be a missing word from a sentence, or the answer to an acronym as in the example below.

  1. Enter your question into the question text box. For example: ‘There's a S.M.A.R.T. way to write management's goals and objectives. What does S.M.A.R.T. stand for in our syllabus?
  2. Then, enter the answer text into the answer box. Highlight any text you wish to be blank and click the ‘Create Blank’ button – In the same way as when creating a Select from List question.
     
    1. When using values that include a comma, please follow the instruction provided and put them in quotations marks using the comma as a separator. For example the cost is "€ 3,49", "3,49", € 3,49".
    1. Once you have added the blank, the highlighted words will be replaced with a highlighted number in brackets. The highlighted words will appear under ‘Blanked words’ below the answer box. This will be the correct answer that the candidate must type into the blank when answering the question.
    2. If needed, you can add accepted alternatives in the ‘Correct alternatives’-box, separating multiple alternatives with commas.
    3. To delete the blanked out word, use ‘Remove’ at the end of the row. To edit the blanked out word, hover over and click on it.
    4. You also have the option to ensure that the correct answer is case-sensitive – tick the box to set this option.
  3. 'Scoring type': this Item type uses Dichotomous scoring by default. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.
  4. Remember to always click ‘Save’ to save and exit the item.

 

Order

The Order question type requires the candidate to put the available answer options into the correct order.

  1. Start by entering your question text into the question box. For example: “”.
  2. Then, enter each of the answers into the answer boxes – there will be 3 available by default but you can add more answers using the ‘Add’ button. Or, if you wish, you can remove answers using the ‘Remove’ button next to the answer you wish to delete.
  3. When you have finished adding the answer options, enter the maximum mark available for the question.
  4. Set the 'Scoring type': Polytomous scoring gives the candidate partial score if answered partially correct. With Dichotomous scoring the candidate must match everything correctly in order to get a score. This Item type uses Dichotomous scoring by default. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.
  5. When you have finished, use the ‘Save’ button to save and close the item.

 

Match

Use the Match question type to allow your candidates to match options using drag and drop.

  1. Enter your question text into the question box.
  2. Enter each option with its corresponding match in the boxes available. There will be 3 pairs available by default – use the ‘Add Pair’ button to add more. Or, use the ‘Remove’ button to remove any matching pairs you no longer require.
  3. Underneath the answer boxes, set the maximum score for the item.
  4. Set the 'Scoring type': Polytomous scoring gives the candidate partial score if answered partially correct. With Dichotomous scoring the candidate must match everything correctly in order to get a score. This Item type uses Dichotomous scoring by default. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.
  5. Click ‘Save’ when you have finished.

Important: you must enter the answers in the correct order when creating a Match question.

If you preview the item, you’ll see that the answer options have been reordered. The candidate must put them back in the correct order by dragging and dropping.

 

Extended Match Question

Use the Extended Match question type to allow your candidates to match options using drag and drop or clicking (if you have a long list and scrolling is not an option). Extended Match allows you to create questions where a answer options can be matched with multiple others or none (so called distractors):

  1. Enter your question text into the question box.
  2. Enter each option with its corresponding match on the right. There will be 3 pairs available by default – use the ‘Add Pair’ button to add more. Or, use the ‘Remove’ button to remove any matching pairs you no longer require.
  3. Click 'Switch mode' to create the right matches/connections. This can be done by drawing a line between or clicking on the circles,
  4. Underneath the answer boxes, set the maximum score for the item.
  5. 'Scoring type': this Item type uses Dichotomous scoring by default currently. Polytomous scoring is on our Roadmap to be developed. Let is us know if you'd wish to use it!

Hotspot

The Hotspot question type allows you to set a question that requires the candidate to click on the correct answer, such as the correct city on the image of a map or the correct colour on the image of a rainbow.

  1. Start by entering your question text into the question box.
  2. Then you will need to select an image from the Media Library or upload a new image. To do this, click the ‘Upload New File’ button. A pop-up will open, allowing you to select an existing image or to upload a new one.
    1. When you have selected your image, it will appear in the item editing screen where you can define the hotspot – i.e. the area that the candidate must click in order to get the marks.
    1. You may draw/add multiple hotspots to the image. When you add a shape, it will appear in red.
    2. The following tools are available for setting up your hotspot:
      1. Pen - to draw the correct area freehand
      2. Rect - to draw a rectangle around the correct area
      3. Ellipse - to draw an oval around the correct area
      4. Clear - to clear any pen marks, rectangles or ovals
      5. Delete to delete the entire hotspot file
    1. The number of hotspots will be listed at the bottom of the editing screen – click the ‘x’ next to one if you wish to delete it. Hovering over it will show which area will be removed.
    2. At the bottom of the editing area, you will see that you have the option to scale the image to prevent scrolling. This may improve the candidate’s user experience depending on the size of the image you are using.
  1. Finally, set the maximum score available for the item before saving and closing them item when you have finished.
  2. 'Scoring type': this Item type uses Dichotomous scoring by default currently. Polytomous scoring can be selected - see our article on 'Polytomous scoring rules'.

 

Short Answer

The Short Answer question type allows you to set a question that requires the candidate to provide a short written answer in response. This question type is best suited for questions that require one word or one line answers.

  1. To create a Short Answer question, simply enter your question text into the question box/ For example ‘What is the name of the legal Act that protects consumers from false or misleading information about goods being sold?’
  2. Enter the correct answers in the answer box.
  3. You have the option of requiring the candidate’s answer to be case-sensitive in order to achieve the mark(s).
  4. Set the maximum score available for the question by entering a value in the box provided.
  5. Click ‘Save’ to save and exit the item.

Note: Remember, the candidate’s answer will have to be exactly the same as the answer set here in order for it to be marked as correct by the system.

Drag and Drop questions

The Drag and Drop question type was developed at the request of a customer for one specific set of questions they had but is now available to all: over a background image candidates need to drag ‘responses’ (other images) into place onto the corresponding placeholders.  Upon creating this question you will see 4 main fields:

  1. 'Enter the question': standard field to enter your question describing the task for the candidate.
  2. 'Add a background image': The background size originally was 700x500 pixels (width x height) with 28,5 pixels per centimetre. The background should not be too large to prevent scrolling issues in delivery. Also: please keep in mind that the background should have room for the list of ‘responses’ / response images - on the right is usually the most user friendly option.
    With your mouse you can mark the placeholders for the responses (Chrome works best for this). Tip: mark placeholders as visual hint for candidates and to help you create the correct placeholders.
  3. 'Add possible responses': This is where you upload the images that represent the correct responses.Tip: responses (images/labels) ideally should be same size as placeholders.
  4. 'Draw connections between placeholders and responses': This is where you match each placeholder with one response. If you check the option 'duplicate responses' you can match multiple placeholders with the same response. This also means that the responses will 're-spawn'. In other words: while answering the list of responses will narrow down ('answers will not be used up') for the candidate, making the question harder to answer.
    Tip: to make the question even harder you can add distractors: responses that are not linked to placeholders.

    2019-01-25_0851-Drag-and-Drop-duplicate-responses.png
  5. Scoring type': this Item type uses Dichotomous scoring by default currently. Polytomous scoring is not yet available. If you need Polytomous scoring developed, please contact your designated super user!

[Example of a Drag and Drop question using duplicate responses in delivery:]

2017-08-02_1223-DandD-delivery_example.png

Tip: PowerPoint can act as simple tool to edit/create images and mark placeholders on your background. In PowerPoint you can right click groups of images and use ‘Save as image’.

 

Sowiso Mathematical question

The SOWISO question type for the exact sciences is an open ended question type that allows the answers to be in the form of formulas. The answers can be scored and checked automatically and can be accompanied with targeted feedback. Moreover, questions can be created with random parameters, that make every question unique.

The Sowiso `Question` tab allows you to edit different placeholders in an exercise:

  1. Question is the text the user gets to see during the assessment
  2. Settings - is where you set the maximum score for this question
  3. Solution is the text the user gets to see after the assessment
  4. Pre and post input are the texts before and after the answer field (for example for x= ... ,  or for a unit, like …. meters).
  1. The Solutions area is where you define what the correct answer is.
    1. Choose an ‘Evaluation type’ for the mathematical area from the list
    2. Use this together with theDefinition field to define the correct answer.
    3. You can add multiple solution rules via ‘Add solution’ to score multiple answers as correct.

When you enter the formulas in a Sowiso maths question, you need to use the appropriate LaTex and Tex or asciimathml delimiters.

Tip: Please refer to our extended manual on Using the Sowiso maths question in Cirrus.

For the commercial details to enable this question type on your environment, please contact us at support@cirrusassessment.com.

 

Comprehensive Integrated Puzzle (CIP) question type

A Comprehensive Integrated Puzzle is a matrix that the candidate needs to complete. It aims to test (clinical) reasoning. In the first column a diagnose, case study or patient subgroup is listed on each row. The subsequent columns are to be filled with various steps in diagnosis and/ or treatment - as categorised in the column header.
The assignment is to fill in the matrix in such a way that each row contains a coherent case.

The author can select dichotomous scoring or choose from several polytomous scoring rules.

Please have a look at our extended manual on the Comprehensive Integrated Puzzle - including a more details and an example of a CIP question.

 The Programming Question type

The Programming question type requires the candidate to write or debug code.
Candidates can be offered the option to run the code or evaluate their code. This evaluation uses testcase(s) defined by the question author. The question is auto-scored based on the number of test cases (written by the author) that are successfully passed.
In order for this question type to work your admins will need to setup a so called execution server at the customer side and some settings in Cirrus.

Read more about creating and using this question type in our extended manual which is available upon request.

The Financial Statement (FS) question type

Using the Financial Statement item type you can create versatile questions specifically geared toward Balance Sheets, Income statements, Account statements.

2019-03-05-Financial-Statement-question.png

Read more about creating and using this question type in our extended manual which is available upon request.

Open Questions

Essay

The Essay question type is an open question type that allows the candidate to provide a longer, free text answer to a question. You can visualise the limit on the number of words the candidate can write and you can upload a marking scheme for the marker – this question type cannot be computer marked.

2017-11-10_1547-Library-Essay-example.png

  1. Enter the question - start by entering the question text in the question box.
  2. Enter the the Marking Scheme - This shows for markers, guiding them how to evaluate and score the candidates answer.
  3. ‘Set the maximum number of words for candidate's answer’ - by entering a value in the box provided. If nothing is entered it will will be '0' meaning there is no maximum. If a maximum is entered the candidate can still type more, but the system will show that the maximum has been exceeded to both the marker and the candidate.
  4. Use the tick box to enable the rich text editor for the candidate to use.
  5. Set the maximum score for the question - change the default value into the max score for the perfect answer (5 in our example). While marking Essay questions assessors are allowed to use decimals, example 1.5.
  6. Click the ‘Save’ button to save the item and preview it via the 'Preview' button.

File Response

The File Response question type allows candidates to upload a file as part of their answer during their assessment. To create a File Response question, start by writing your question text in the question box.

  1. Next, enter the the ‘Marking Scheme’ for the marker – this question type cannot be computer marked.
  2. Finally, set the maximum score available for the question in the field provided and click ‘Save’ to save and exit the item.

Note: Be aware that your candidates need to be able to access their file for uploading. If they are taking high-stakes summative assessments, this might be disabled for security reasons - i.e. they won’t be able to access storage media (like an USB drive) or online sources.
You can also provide a document for the candidate to download first, for example, you could attach a predefined spreadsheet for the candidate to complete and upload - if security settings allow for downloading and uploading of course. Also be aware that candidates might not have access to the appropriate software, for example to edit an Excel-sheet.

Additional Item types

Welcome pages

The welcome page is the 'cover page' of your assessment. It typically covers a word of welcome and instructions for you candidates before they click on start and their assessment timer start to run.

An example of a welcome page as being edited by an author in the Library:

2019-09-17_Library-Welcome-page-example.png

  • A 'welcome screen' is needed for each assessment - read how to add it to your assessment.
  • You can create a unique welcome page for each assessment or reuse a generic welcome page for multiple assessments.
  • Create them in any collection in your Library by clicking 'Add Item'-button > select 'Page'. On the next screen simply give it a title, add page content and
  • 'Select the page type' 'Welcome screen', select status 'Live' in the right hand column so it can be used in assessment like any item and click 'Save and Close'.
  • Optional: you can add a PDF or large image as a resource or media file to the finish page.

Finish pages

The finish page is the final page of your assessment, which only shows after the candidate has handed-in their exam. This page is show directly after the candidate has submitted their assessment. It typically covers instructions on what is next. Whether this is clicking the 'Next' button to see their results, for example in their Score Report. It could also be a simple message like "Thanks for your participation. Our organisation will share your official results with you in 5-10 business days."

  • A 'Finish screen' is needed for each assessment - read how to add it to your assessment.
  • You can create a unique finish page for each assessment or reuse a generic finish page for multiple assessments.
  • Create them in any collection in your Library by clicking 'Add Item'-button > select 'Page'. On the next screen simply give it a title, add page content.
  • 'Select the page type' 'Finish Screen' and select status 'Live' in the right hand column so it can be used in assessment like any item and click 'Save and Close'.
  • Optional: you can add a PDF or large image as a resource or media file to the finish page.

Other screen

When you click 'Add Item'-button > and select 'Page'. On the next screen you may notice the option to 'select page type' > 'Other screen'. This option allows for certain bespoke features for certain customers and is generally not used.

Non-Disclosure Agreement (NDA)

When delivering assessments, you can choose to serve a Non-Disclosure agreement which needs to be accepted before you can start the assessment. This page is not maintained in the Library, but setup as a template in the Admin section. Read how to set this up in our article 'Adding a Non-Disclosure Agreement to your assessments'.

Survey pages: the registration form

Sometimes it is essential to collect extra information from your candidates. You can setup a survey - called a registration form in Cirrus - to be filled out before or after taking your assessments. This can help you to:

  • Collect information on your candidates - for example Awarding Bodies that need to collect registration details from their candidates.
  • Collect information on your assessment - you can ask you candidates about the assessment or even certain items within the assessment.
  • Collect information on your educational program  and it's tutors / trainers - if you are not just assessing them, but also teach them: why not ask about how well they felt prepared or how you can improve your program?

These surveys are not maintained in the Library, but setup as a template in the Admin section. Read all about it in our article 'Registration forms: collecting information before or after the assessment'.

Section Pages and Timed Sections

With a "section page" you can describe cases and deliver them in conjunction with specific questions about that case. A section page is best created in the same collection as the correlating questions. You can then add it to your assessment when you generate this. Read more about  Using (Timed) Sections in your Assessment.

...

Comments 0

Cirrus uses Zendesk to provide our Help Center services. Check out Zendesk's Terms of Service and Privacy Policy, as they govern your use of these services.

Powered by Zendesk