4. Assessment Form - building, previewing and publishing it

Cirrus Assessment Support Team -

Step Four – Building your Assessment Form before previewing and publishing it

In step 4 on the 'Generate Assessment'-tab you will build your assessment form (i.e. the exam script for your candidates, preview and publish it. The steps involved:

Creating an Assessment Form

2017-07-05_1652-assessment-generate-form.png

On the 'Generate Assessment'-tab you first create an assessment form (creating multiple assessment forms is on our roadmap for later). 

  1. Click 'Add form' to create a new form. You will be prompted to choose the amount of questions to be used in your form out of the total selected questions available and need to choose the type of assessment to be Fixed or Random / LOFT. Read more about this further down.
  2. You can share, publish and preview your assessment form from this screen.

Upon creating an assessment form you will be presented with two pivotal choices in the 'Generate assessment modal window':

2017-05-19_1044-assessment-generate-modal-window.png

 

Defining your type of assessment: Fixed or Random/LOFT and the number of questions to be used

The first choice is the amount of questions you want to use from your total pool of selected questions. As a general rule of thumb for manually assembled assessments you will choose all (i.e. 50 out 50 questions for example). However: when using a large pool you can opt to use 50 out of 200 questions at random.

The second choice is between a fixed of a randomly generated assessment: 

  • 'Fixed assessment' - the Assessment will be generated in advance. All candidates will get the same set of questions.
  • 'Randomly generated assessment'. If you select randomly generated assessment, the test will use Linear On the Fly Test generation (LOFT) which means each candidate will get a unique set of questions

For example:

Fixed 50 out of 100 means all candidates will get to same set of 50 questions. (Whether or not they are in the same order is determined by the Assessment Option 'Question navigation - Display questions in random order). 

Random 50 out of 100 means that that each candidate would get 50 questions out of those 100 at random. In this automatically means that the questions will be different and cannot be in the same order.

As a general rule of thumb for manually assembled assessments you will choose all (i.e. 50 out 50). However: when using a large pool you can opt to use 50 out of 200 questions at random in such cases.

After you click the ‘Create’ button, the screen will refresh and you will see the assessment items listed.

Building your Assessment Form: ordering and adding sections

After you have generated your assessment form according to the steps listed above, the screen will refresh and you will see the assessment form listed. Click on it an your assessment form will open -This screen is packed with information and possibilities!

2018-07-05_1634-Assessment-new-form-first.png

1. Assessment form: Introduction

In your assessment form clicking on 'show introduction' will expand a list for showing / adding any pages before your candidate starts:

2018-07-05_1656-Assessment-form-introduction-options.png

  • Welcome screen - this is the introduction text for candidates before they start their assessment. This can include the assessment title and any information and instructions (on allowed materials for example)  you wish to share before candidates start. [required]
  • NDA: decide if you want to add a Non-Disclosure Agreement.
  • Registration form - use this survey your candidates in order to collect additional information or feedback before or after their assessment. See our article on using the registration form in Cirrus

2. Assessment form: Questions

In your assessment form clicking on 'show questions' will expand the page to show questions and sections to be delivered to your candidate:

2018-07-05_1711-Assessment-form-Questions-options-no-numbers.png

  1. 'Include Items' and Exclude items buttons - use the checkboxes with these buttons to easily include or exclude items to be delivered. 
  2. Add Section - see our article on Using Sections in your Assessments.
  3. Use the Search-box to find a specific question by title or question ID.
  4. Included questions with a check mark will be presented to your candidates. Excluded questions without this check mark will not be used in this assessment form. This also gives you the option to remove a question should you need to: just exclude it.
  5. Overview listing Questions text, ItemID, Question type, Last Modified date of the Item, Learning Objectives, Score and Status + a 'quick item preview' button.
  6. Preview / Versioning - the [...] button gives access to the Assessment Preview and allows for Assessment Versioning.
  7. Go back using the browser or the breadcrumb if you
    1. If you need to include a new or forgotten question to your form use the 'back button' or click on the assessment title in the breadcrumb to go back to the question selection tab.
    2. You can also completely reset your form and start over by deleting the form and creating a new one in the previous screen.
  8. Share - simply use the 'Share button'  to share your assessment form with co-authors. As with other content the author of an assessment needs to actively share it to be accessible to any other user.
  9. Publish -simply go back one screen (using the back button) and use the 'Publish-button' to publish your assessment which will make it available for Scheduling and thus available for delivery it to your candidates.

3. Assessment form: Finish

In your assessment form clicking on 'show finish' will expand a list for showing / adding any of the following pages after your candidate has handed in:

2018-07-05_1656-Assessment-form-finish-options.png

  • Finish screen - you will need to select a finish page for your assessment. This page is shown directly after the candidates have submitted their assessment.
  • Registration form - use this survey your candidates in order to collect additional information or feedback before or after their assessment [link]. 
  • Other - info coming soon.

Please note: depending on the assessment settings this can be followed by feedback for the candidate.

Ordering: you can reorder the items in your assessment by dragging them up and down.

Please note: the scenario described above is for a manually assembled assessment using a fixed amount of items. For randomly generated assessments this work differently. Alternatively you can also use Blueprint question selection for an assessment.

 

Assessment Form > Options: determining Grades and Pass Mark for your assessment by assigning the Assessment Scale to be used

The step below is the way for authors to set the passmark and grade scale used for their assessments.

A very important step when creating your assessment is choosing which assessment scale will be used. (Also known as Grade Scale or (in Dutch) Beoordelingsschaal of Cijferschaal.

 2018-05-30_1711-assessment-form-options-tab.png

When you are editing your Assessment From you click on the option tab to:

  1. Select the Assessment Scale you want to use for with your Assessment Form. 
  2. Grade - the table will show the highest grade, the pass mark grade as setup in the Assessment Form
    From and Including % - this shows the values set. The value for the Grade set as the Pass Mark can be changed - see below.
    From and Including Score - this shows the scores needed based on the max score of your assessment form. 
  3. Click 'View complete assessment scale' to see all grades listed
  4. Adjust percentage to pass for chance score - the automatically converts the grade scale and pass mark based on the chance score for your assessment form. In order to use this you need to fully understand the implications.

Please note:

    1. it is imperative that Assessment Scales are always setup taking into consideration the Rounding Rules for your environment
    2. Assessments use a local copy of scales defined under Admin > Assessment Scales. If the generic one is updated be sure to inform you authors and have them reattach it to their assessments if applicable.
    3. Assessment Authors can change the Pass Mark value for the particular Assessment Form they are in. The other Grade before and and after the Pass Mark will be recalculated automatically when this is done and redistributed evenly (linearly).

 

Dividing questions into Sections in your assessment

In Cirrus you can easily add Sections: context blocks with associated questions. Sections are sometimes also known as a 'casus',  usecase or stimulus pane. You create the specific case text (with options for images or video) in your Library like any regular Item. In this step you add it to your assessment and assign specific questions to it, thus creating a section. 

Here is an example of a section as viewed by candidate in delivery:

 

 

Read more: About using Sections and Timed Sections in your assessments.

Remember: in the Library you can add Section pages which form the basis for sections in your assessments. Section pages contain the text for your section, a paragraph related to multiple questions for example.

 

Assessment Sharing: working together on assessments and anonymous sharing

To work together on assessments you can simply use the 'Share button' to share your assessment form with co-authors. As with other content the author of an assessment needs to actively share it to be accessible to any other user.

The permissions related to working on an assessment are centrally defined and managed under Admin > Roles > Assessment (roles). By default the 'Author' can share assessments but not the 'Co-Author' who in turn can see and do more than a 'Reviewer'. 

Anonymous sharing - a special form of sharing of any assessment for demonstration purposes via a simple link. 

Using our 'Anonymous sharing'-option and an URL will be generated. Share this link with anyone and they will only have to click it to try out the exam immediately: no login is needed. The UI will use the default language as setup in your environment.

Please note that there will be a maximum of 50 sessions per month for this feature and no results will be recorded for these sessions. Please note that *anyone* using this link can view your questions!

Previewing your Assessment

Previewing your Assessment is a crucial step in before making it available to your candidates:

The Author Preview - use this to check all settings and introduction and finish texts.

The Candidate Preview - use this to check the assessment from a candidate's perspective.

Optional: (temporarily) enable an anonymous sharing link to preview your assessment

Stick to the four-eyes principle: get someone to review this with you!

You click on [...] and use the ‘Preview’ button in the top right of the 'Generate Assessment -tab' when editing an assessment. This will open the preview screens:

Preview: the Author overview

The Author overview when previewing an assessment shows one page listing all the assessment's settings, information, the candidate experience settings. It also lists the total maximum score and all items in the exam plus their associated Learning Objectives and Taxonomies:

2017-05-19-assessment-author-preview.png

Preview: the Candidate preview

Click on the Candidate-tab to start the 'Candidate'-preview. This enables you to view the assessment 'as the candidate would see it' - minus the header bar. As the author you can preview as caniddate and take the exam from start to finish. This should be your final check before publishing the exam and scheduling so your candidates can take it.

It enables you to preview any NDA, Survey and the start screen of the exam. You also get to see the questions.

  1. the correct answer options is marked in green (this is not visible to candidates of course)
  2. Click on 'Show item information' to see information on Score (the maximum score for the question), Seed Item (yes or no), Learning Objective(s) for the question and the Item ID. This is also not visible to candidates)
  3. Navigate as a candidate using the navigation bar at the bottom.

Here is an example of a multiple choice question in the candidate preview:

2017-08-07_1116-assessment-candidate-preview-example.png

Here is an example of the same page in delivery during the actual exam:

2017-08-07_1116-assessment-candidate-delivery-example.png

Here is an example of a section as viewed by candidate in delivery:

Publishing the assessment - so it becomes available for scheduling

If you've setup your assessment following all the steps, by the time you click on 'Publish'-button you usually only have to select a marking workflow and put the status to 'Live'. However, a handy 'To do' list shows you which steps you are still required to take:

2017-12-12_1009-publish_assessment_example.png

  1. Select the marking workflow for your assessment
  2. Set the 'Assessment' to 'Live'
  3. The 'To do list' will show which steps are required before you can publish.

If all actions are complete, hit ‘Publish’  - Publishing the assessment will mean that it is available to be scheduled for candidates.

Now you are finished creating your assessment and it is available for Scheduling to deliver it to your candidates

Please note: 

Editing, Archiving, Deleting an Assessment that has been used

Editing an assessment that has been used

You can move an assessment back to "Draft" as long as it has not been Scheduled.
Assessments that have been scheduled and or have been taken by candidates cannot be edited. In this case you can:

  • Create a new version of your assessment (when you add/remove a few questions)
  • Make a copy and edit it to your liking before redistributing it again (when you use completely new/ different questions).

How to delete or unpublish an Assessment that has been used?

Remember: to make an assessment unavailable for scheduling you can either unpublish it (if that is still possible) or archive it. This is done on the assessment overview page by checking the checkbox in front of it and hitting "Archive". Deleting assessment should not be possible and is not an option once it has been used / taken by candidates and will be kept for reference to these deliveries.

 

Comments 0

Cirrus uses Zendesk to provide our Help Center services. Check out Zendesk's Terms of Service and Privacy Policy, as they govern your use of these services.

Powered by Zendesk