User Manual
- Accessing the Assessment Builder Module
- Building a Questions and Answers Database
- Creating Assessments
- Clicker Assessments
- Assessment Reporting
- Further Assessment Considerations/Information
Accessing the Assessment Builder Module
Introduction to the assessment builder module
This document focuses on the Assessment Builder module within the Signify HR System, which creates online assessments to evaluate an employee's knowledge of a specific subject. These assessments can then be added to pathways within the Signify HR System and made available to certain employees as required. Upon reaching the end of the document, a person should be able to understand the positioning, the purpose and application of this module. General users should be able to navigate themselves through an assessment.
Accessing assessment builder
Assessment management screen
The primary Assessment Management Page which opens from the above menu option has another menu at the top from which all the various screens can be accessed containing the Assessment Builder functions. The basic functions contained in these menu options, to be explained in more detail in further sections of the document, can be summarized as follows:
- Creating answer groups which contain standard answers which will be used in questions.
- Creating a database with questions and answers to be used in creating assessments.
- Creating an assessment by using the questions in the questions and answers database.
- Importing an assessment from an MS Excel spreadsheet workbook.
- Printing reports related to the assessments and questions in the database.
Building a Questions and Answers Database
In order to create an assessment, one needs a database of questions with their answers - which can be used to compile the assessment. The same question may be used in more than one assessment, and if it is changed, the change will be visible in all assessments which contain the question. To build up the database, one can create answer groups which can be re-used, questions and answers for each question in the database.
Populating the Answer Groups Database
This step is optional but could help to considerably speed up the building of assessments. Many questions have similar answer structures e.g.:
- Excellent / Good / Poor
- Yes / No / I don't know
- True / False
- Not applicable / I strongly disagree / I disagree / I neither agree or disagree / I agree / I strongly agree
- Not Applicable / Poor / Fair / Average / Good / Excellent
If these can be placed into an answers database, they can be reused where appropriate. To see the Assessment Builder menus, from the main menu select Modules | e-Learning | Learning Management | Assessment Builder. To add standard answers, select Answer Groups from the Assessment Builder menu.
List of Answer Groups Screen
The main screen which opens for answer groups displays a list of all existing standard Answer Groups in the system database.
- To create a new Answer Group click the Add Group button.
- To view or edit the list of standard answers per Answer Group, click View/Edit next to the group which you would like to view or edit.
- To delete an Answer Group, click the checkbox in the Delete column and then click the Delete button.
Adding/Editing an Answer Group
When adding a new group, viewing or editing an existing group, please note the following:
- Provide a descriptive name for the group. We like to include the number of answers in the answer group description.
- When creating a new group, click Save Answer Group Details to create the group before adding all the possible answers to the group. Then the button with which to add answers to the group becomes available.
- Click Add Answer for each possible answer which must be added to the group.
- To edit a specific answer, click on the Answer.
- To delete a specific answer, select the checkbox in the Delete column and then click Delete Answer(s)
Adding/Editing Standard Answers in an Answer Group
When adding a new answer to the answer group or editing an answer in an answer group, please note the following:
- Complete the required fields on the Answer Detail screen and click Save to submit the answer details and still display the answer details on the screen after saving.
- Alternatively you can click Save Add New to save the answer details and open another blank screen where the next answer in the group can be provided - this helps to save time when adding multiple answers.
- Continue adding the answers until the group is done, after which you can click on Back to go back to the list of answer groups.
Properties which can be captured per answer:
- Answer: The potential answer which must be displayed
- Standard Feedback: Automatic feedback which the system provides directly after the question has been answered, if this answer was chosen, regardless of whether the question was answered correctly or not.
- Correct Answer Feedback: Automatic feedback which the system provides directly after the question has ben answered, if this answer was chosen and the question was answered correctly.
- Incorrect Answer Feedback: Automatic feedback which the system provides directly after the question has ben answered, if this answer was chosen and the question was answered incorrectly.
- Score: The score to be allocated to the answer if chosen by the user when completing the question.
- Display Order: The order in which the possible answer will be displayed to the user when completing the question - if all answers have the same order (e.g. zero), the answers are displayed in random order for each completion of the question per user.
Please note that all of the above properties are also available per question after adding them to a question in the database and can easily be edited per question, once added to the question.
Populating the Questions Database
To simplify the creation of an assessment, it is advised to start with populating the questions database (once all possible standard answer groups have been captured) so that, when the assessment is created later on, there are questions available to choose from.
- To see the Assessment Builder menus, from the main menu select Modules | e-Learning | Learning Management | Assessment Builder.
- To add questions and question categories, select Question List from the Assessment Builder menu.
Question Management Screen (Question Database Tree View)
On the Question Management screen you will be able to add new question categories, edit existing question categories and add/edit questions. The questions are organized in a tree view which can be expanded per category/branch (by clicking on the "+" to the left of the category) to display all the items contained in the category. Each category can contain other sub-categories and/or questions. The buttons on the screen will become available per node within the tree structure which is currently selected - e.g. with "eAssess Questions" selected, only the Add Category button is available.
Adding a Question Category
Use a Question Category to achieve the following:
- Group similar questions e.g. questions about a specific topic.
- Group questions that will be used in one assessment (although this makes reusing these questions in other assessments less intuitive)
- Group questions according to their types e.g. multiple choice, true or false etc.
Further:
- Sub categories can be added below a category to group questions even further.
- To add a new category click the Root node of the Question Management tree view (eAssess Questions): The Add Category button becomes active.
- Click Add Category for the Question Category Detail window to display.
- Capture the category name/description and click Save.
- Once a new category has been saved, the Categories and Questions tabs become available.
Adding a Question to a Category
In this section we will proceed to add questions to a question category. Questions are built up via three sections for each question. They are:
- The question itself. (Provided on the Question Detail tab)
- Potential answers for the question. (The Answers tab)
- Feedback to be given to the user when the question is answered. This could take the form of a standard message, a message given only when the answer was correct or a message in case of an incorrect answer. (The Feedback tab)
On the Question Category Detail page click the Questions tab and then the Add button for the Question Detail window to appear, as shown below. On the Question Detail tab fill in all the relevant fields.
Capturing question details
Properties which can be captured per question:
- Introduction: Any introductory message that is to be shown to the user before displaying the question - normally some background information to the question.
- Question: The text stating the actual question to be displayed in the assessment.
- Type: The Type of Question - explained in more detail below.
- Question Score: The total score which can be scored in the question if answered correctly. In the case of a Multiple Select type question, this should be the total of all possible correct answers. If this is set to zero, the system will automatically calculate the total score as the sum of all correct answers.
- Maximum Answers Allowed: Works in conjunction with question Type. If the question Type is set to Multiple Selection, this field specifies the maximum number of answers that can be selected by the user (if set to zero, all possible answers can be selected). For other question types, this value is ignored.
- Show Comments Block: If this checkbox is checked, an additional text box will be displayed directly below the question answer where the user can capture comments (perhaps to explain/motivate an answer).
- Comments Mandatory: If this checkbox is checked, the Comments field becomes mandatory for the question, forcing the user to complete the field for the question.
- Comments Heading: The heading or label to be displayed directly above the Comments field.
The question Type can be one of the following:
- Single Selection: The question has one possible correct answer from which the user can choose using a radio button group - this is the default value and most used in assessment questions.
- Multiple Selection: The question has multiple correct answers which the user can select using checkboxes - e.g. 5 possible answers of which 3 are correct.
- Match Column A with B: The question has a list of values in one column and the same number of correct answers to choose from in a second column.
- Select the missing word: A sentence or paragraph is displayed for the question which contains at least one missing word, each missing word having only one correct answer and can be chosen from 2 or more options.
Once all details for the question as a whole is completed, click Save And Go To Answers: The Answers and Feedback tabs become available and you are taken to the Answers tab to add possible answers to the question. Guidelines per question type are given below, further explaining how the answers to these questions should be added.
Adding Answers to a Question
For every possible answer to a question, click Add Answer to open the Answer Detail screen, where the answer details are captured.
Capturing Answer Details
In the Answer Detail screen capture the details to the answer and click Save. To save time, you can capture the details for an answer and click Save Add New, which saves the answer details and opens a new blank page to add a new possible answer.
Properties which can be captured per answer are the same as for answers in an Answer Group:
- Answer: The potential answer which must be displayed
- Standard Feedback: Automatic feedback which the system provides directly after the question has been answered, if this answer was chosen, regardless of whether the question was answered correctly or not.
- Correct Answer Feedback: Automatic feedback which the system provides directly after the question has ben answered, if this answer was chosen and the question was answered correctly.
- Incorrect Answer Feedback: Automatic feedback which the system provides directly after the question has ben answered, if this answer was chosen and the question was answered incorrectly.
- Score: The score to be allocated to the answer if chosen by the user when completing the question.
- Display Order: The order in which the possible answer will be displayed to the user when completing the question - if all answers have the same order (e.g. zero), the answers are displayed in random order for each completion of the question per user.
Guidelines: Single Select Questions
Guidelines: Multiple Select Question
Guidelines: Match Column A to B
Guidelines: Missing Word
Guidelines: Text only (Manipulated Single Select)
Adding all Answers from an Answer Group to a Question
Creating Assessments
Creating an Assessment Training Intervention
Assessment Management Screen (Assessment Database Tree View)
The primary Assessment Management Page which displays a tree view of Assessment Categories and Assessments which are present in the database currently.
Adding an Assessment Category
Adding a sub-Category within an Assessment Category
Adding an Assessment to an Assessment Category
Adding a Section to an Assessment
Creating/Editing an Assessment Section
Adding Multimedia to an Assessment Section
Linking Questions to an Assessment Section
Adding Multimedia to a Question within a Section
Linking a Question Category to an Assessment Section
Clicker Assessments
Clicker Assessments
Document Purpose
Aim
The aim of this document is to provide a high-level overview of the Signify Clickers Sub-Module.
Clickers are used by learning administrators for the real-time assessment of groups of learners who do not necessarily have access to computers or the literacy to answer online assessments. It simplifies the assessment and allows each individual to provide his/her own unique answer.
Clickers
Introduction
The Signify system has been enhanced to integrate with the ‘ResponseCard RF LCD’ clickers from Participate Technologies. The functionality allows learners to complete assessments in groups, without having direct access to a computer.
A facilitator will assign clickers to learners and each learner will be able to answer questions read to them by the facilitator. The computer then stores the last input on each question from the learner’s clicker. After completion of the assessment a learner record is created for each learner and the assessment is marked.
The following will be discussed:
Process Flow
The following sections will discuss the setup of clickers, creating a clicker assessment, the administration of a clicker assessment, the learner’s perspective and learner records
Setup of Clickers
Two processes have to run simultaneously on a computer for the clickers to work.
The first is the Clicker module from the Signify LMS, the second is the Clickers Zip File. The zip file can be obtained from Signify. Once the file has been received, ensure that it is not in Zip format and that it is saved on your computer where it can easily be accessed.
If the folder was saved as a Zip file, unzip the folder by clicking on the right button of the mouse whilst the cursor rests on the folder. This will load all the file options.
Click on “Extract All” to move the files from the Zip folder to a different file. Use the default location or alternatively specify a new location by clicking on the “Browse” button.
Configure Clicker Relay
The Clicker Relay allows the LMS to connect to the Clickers. It is similar to a language translator and it will translate the input from the clickers to responses on the LMS System. Two processes need to be configured beforehand and run simultaneously for the clicker assessments to work.
First Configuration Process
In the Signify LMS System, go to the Learning Configuration Module Settings Detail page.
System Administration | Tools | Configuration | Framework | Human Resources | Learning.
Scroll down to the “Learning - The URL of the Clicker Server Relay component.” There should be a value in the text box provided. This is called the “URL of the Clicker Server Relay Component.” This component should be pre-configured, however, the URL (e.g.: ws://127.0.0.1:8081/) should be noted for the next step. If the text field is not configured, please contact Signify Support for assistance.
The second step in this process is to activate the module.
System Administration | Tools | Configuration | Framework | Portal Web Part Administration
Scroll to “Clicker Assessments” and ensure that both the “Visible” and “Enabled” tick boxes are selected.
Second Configuration Process
Locate the unzipped Clickers folder and open it. There should be a list of different file folders, XML documents, CONFIG files and different Manifest files.
*Please note that if any of these files are amended incorrectly, the clickers functionality is unlikely to work.
To enable the clickers, find the below 2 files:
- “SignifyHRClickerServerConsole.exe.config”
- “SignifyHRClickerServerConsole.vshost.exe.config”
The files can be found here: SignifyHR.Clickers.ConsoleApplication | bin | Debug.
These files will be configured using Notepad. To edit these files right click on the file and select "Open with" then select the Notepad program.
Once the file has been opened in Notepad, find “<add key="WebSocketServerPortNumber" value="8080"/>” and change the numeric value (e.g. “8080”) to the URL numeric value described in the 1st Configuration Process. Therefore, if the value from the 1st configuration process is to be taken the new value would be “<add key="WebSocketServerPortNumber" value="8081"/>”.
- Click on “File”, then on “Save”
- After the change has been saved, click on “File” then on “Exit”.
This process must be completed for both files.
Complete this process with both of the files:
- “SignifyHRClickerServerConsole.exe.config”
- “SignifyHRClickerServerConsole.vshost.exe.config”
Hardware Configuration and Check
After the configuration folders have been saved, insert the “Turning Technologies” dongle into any of the computer’s USB sockets.
Thereafter go back to the Clickers folder (SignifyHR.Clickers.ConsoleApplication | bin | Debug) and run the file named “SignifyHRClickerServerConsole.exe” as administrator.
To run the program as administrator right click in the file and select "Run as administrator".
This opens the console window and lists the processes. Each process should have “Done!” listed next to it.
- Connecting to hardware...
- Validating the hardware license...
- Initialising receiver...
- Initialising polling session...
- Initialising Web Socket Server...
The last line should read “Listening for web socket connection…” To test that the dongle can receive signals from the Response Card, click on any number of any of the response cards. A message in green will display: “*** Web Socket: Message failed to send, no connection active:”. This message indicates that the Response Card input was received by the computer.
If any of the actions have failed kindly contact Signify support to start the troubleshooting process. support@signify.co.za
Create a Clicker Assessment
To create a clicker assessment, please refer to the Assessment Builder Manual. Note the points below.
- When creating the assessment, the Assessment Type in the Assessment Detail window must be Clicker Assessment
- Only Single select questions must be used.
- The assessment questions can be exported by clicking on the “Export to Excel” button. The exported file can be printed for the facilitator to keep a copy.
Assigning Facilitators
The default configuration of the Signify system has a target audience set on the Clicker Assessment menu item. This is set so that only users with the role Assessment Facilitator will be able to access the Clicker Assessments.
To assign the role the the user navigate to:
System Administration | Users | Manage Users.
Search for the user you would like to grand the Assessment Facilitator role to. Then select the user or Click on Action | View/Edit. Select Role assignment on the left hand side menu then click on "Add Multiple Roles".
The facilitator might have to log out and back into the system for the changes to take effect.
Facilitate a Clicker Assessment Session
After the configuration has been completed and a Clicker Assessment has been created, the Assessment will be made available in the Clicker Assessment list.
Follow the path below to access the Clicker assessments.
Home | Clicker Assessments
Remember that the Console has to run at the same time as the Signify LMS system in order for the clickers to work. If the Console is closed at any time, the clickers will cease to work.
Once the above path is followed, the Learning page will load with all of the available assessments.
- The search function can be used to find a specific assessment.
- Click in the Text box and type the assessment name or part thereof to search for a specific assessment.
- Click on the "Search" button to search for the assessment.
A specific assessment can also be found by scrolling through the available assessments.
Clicking on “Start” will load the selected Clicker assessment.
If the Participate Technologies Dongle is disconnected, or the “SignifyHRClickerServerConsole.exe” is not open/configured, the screen will flash between “Connecting” and “Disconnected”. If the Console is open and the Dongle inserted, click “Retry”.
Once the clickers are linked to the LMS system, the “Clicker Receiver Details” page will load. The status will change to “Polling”.
Together with the receiver details, the Response Card Assign window will load. This window is only activated once any of the Response Card’s buttons are pressed.
The facilitator will be able to select a preferred subgroup of users that they would like to assign to clickers. To do this select the hand icon and search for the subgroup you would like to select from.
Only users in the selected subgroup will be available for selection when assigning a clicker to them
- Device ID: The device’s unique identification will be loaded automatically when the response card is registered on the LMS. Each device’s unique number can be verified by looking at the back of the card.
- Employee: Type the employee's name, or Surname, or learner number in the text box to find them on the system. To view the first 30 users in the selected subgroup insert a space in the employee field.
- Refer to the Manage Employees Manual to add learners
- Status: When the response card is activated on the system for the first time, the status will automatically be “unassigned”. As soon as a learner is linked to a response card, the status will change to “enabled”.
- Response: The specific response used by the learner will be noted in this column.
- Assign: Once a clicker has been linked to a learner, click on “Assign” to link the specific response card to a learner.
- Cancel: Click on “Cancel” to remove the activation option for the response cards.
- Action Disable: A card can be disable by clicking on “Actions” and then on “Disable. This will disable the learner’s clicker without removing the clicker from the learners’ name. The status of the clicker will change from “enabled” to “disabled”.
- When a card has been disabled, it can be assigned to another learner by clicking on any of the response Card’s buttons. This will load a new clicker receiver details row. The same process as above can be followed when assigning the clicker to another learner.
- Action Enable: If a card has been disabled, and not re-assigned, it can be re-enabled by clicking on “Actions” and then on “Enable”. This will re-enable the learner’s clicker.
- Action Re-Assign: To reassign a new clicker to a learner, the current clicker has to be “Disabled” first. Once a clicker has been disabled, click on “Actions” and then on “re-assign”. Then click on any of the buttons of the new clicker, to assign the new clicker to the learner. After the Clicker has been linked, click on “Enable”, to provide the learner usage of the clicker.
- Continue: After all the clickers has been assigned, they will be listed in the Clicker Receiver Details Window. Click on “Continue” . This will load the Clicker Assessment.
The Clicker Assessment Window comprises of 2 separate sections on the same window.
The first section is the question section, the second is the “Clicker Receiver Details” section. The questions for the clicker assessment are compiled in the assessment builder. They will appear in the window for all of the learners to see.
The Clicker Receiver Details window will have all the assessment details as well as other functions for the Assessor to use. The below information will automatically appear. It is used primarily to provide information about the assessment.
- Font Size
- Status
- Hardware ID
- Model
- Firmware Version
- RF Channel
- Session ID
- Session Start Time
- Facilitator.
Font Size: This allows the facilitator to change question and answer size on the screen.
Responses: The number of individuals who have responded will be listed. A “number” out of “total” value is provided. This allows the facilitator to gauge how many responses are still required for a 100% response ratio.
Click on the colored icons next to each section to expand the section. The window will expand with each individuals’ details and their progress. The Response per question section will provide the facilitator with information regarding the overall progress per question.
- Status: The status will show each learner’s clicker states i.e. Enabled or Disabled. When an individual’s clicker is disabled, the responses number will adjust automatically and the number of inactive learners will be indicated at the top.
- Employee: Each learner with their Employee Number will be listed next to their status.
- Device: Each device ID will be provided next to the corresponding learner.
- Responses: There are two responses recorded.
- Responses in Green indicate the number of valid (Both correct and incorrect) responses provided
- Responses in Orange indicate the number of invalid responses.
- Cross/Tick: A cross indicates that the specific learner has not yet responded, a tick indicates that a valid response was provided.
Questions: The LMS also provides a full overview of all the questions. Click on the down arrow to view the overall progress.
- Question: The question number is provided in this Column. Click on the question number to navigate to the question.
- Attempted: Green indicates that a question has been opened, red specifies that a question has not been seen by the learners.
- Answered: Green will indicate that all of the enabled learners have provided valid answers. Red labels show that there are responses owing.
- Last answer valid: green indicates that the final answer provided by all of the learners were valid, red indicates that a valid answer was provided, but then changed to an invalid answer.
Click “Edit Participants List” to navigate back to the Clicker Receiver Details page.
- All precious responses will be recorded; however, the current questions’ responses will not be recorded.
- Learners can be disabled prevented from continuing at any point.
- Learners can be enabled at any point
- Clickers can be re-assigned to learners at any point
- New learners can be added at any point.
To move to the next question, click on the “right arrow” , and similarly, click on the “left arrow” to navigate to the previous question. When clicking on any of the arrows, the LMS will show a popup screen requiring the facilitator to confirm that he/she want to move the next/previous page.
- Click on “Yes” to continue to the next question
- Click on “No” to return to current question.
Note the below when completing assessments:
- Assessments can be completed with disabled learners
- When moving to the previous question, the current question’s responses will not be recorded. The responses from the previous question will load with the question.
If one or more candidate’s selection is still outstanding, the system will display a different message and list each learner who have not yet provided a response.
- Click on “Yes” to continue to the next question
- Click on “No” to return to current question.
Once all the questions have been completed, a “Finish” button will display instead of “right arrow” button. Click on the “Finish ” button to load the “End Assessment” notification screen.
- Click on “Yes” to finish the assessment.
- Click on “No” to return to the assessment.
Regardless of whether a question is marked as mandatory or not in the Assessment Builder, the LMS will prevent the administrators from completing the assessment if all the questions have not been completed by all the learners. If there are outstanding responses, a message will display and inform the administrator that not all questions have been answered by the enabled participants. Click on “Ok” to return to the question list and use the Questions List to find the missing responses.
After all questions have been completed and the “Finish” button has been pressed, the “Assessment Complete” screen will load.
Click on “Results” button to view each learner’s results. The results screen will open up.
- Employee Details: Click on employee details to sort the list according to the learner’s names and surnames
- Score: Click on “Score” to sort the list from highest to lowest, or from lowest to highest.
- Passed: Click on “Passed” to list the learners who have passed first, then according to their names and surnames.
The assessment will be completed and learner records will be available for all the participants.
Learner's Perspective
The Clicker Assessment module does not require learners to access the system. Their interaction is limited to the Turning Technologies clickers and the questions that are presented on a projector. It was developed for ease of use and is ideally suited to situations where learners only have basic or lower computer literacy levels. It can also be used with learners who have profiles, but do not have access to the system.
Learner Records
Learner Records are created for each learner after they have completed the clicker assessment. These can be accessed form the Learner Records Menu (for a full discussion of learner records, please review the Manage Employees Manual).
Learner Records can be accessed from the Employee Development section on the right-hand side of the LMS window.
Manage Employees | Find Employee | View/Edit | Employee Development | Learner Records
Click on “Print Learner Record (Classic)” or “Print Learner Record (Standard)” to load the print learner record screen.
If “Print Learner Record (Classic)” a popup screen will open with all of the individual’s learner records. Click on “Print” to expert the learner records to PDF. The available folder will be available in the Downloads folder on the computer from where the records were generated.
Click on “Close” to close the window and return to the learner records.
If Print Learner Record (Standard)” is clicked on, a popup screen will open with all of the individual’s learner records. Click on “Print” to export the learner records to PDF. The available folder can be found in the Downloads folder on the computer from where the records were generated. The window can be closed by clicking on the “Close Window” button in the top right-hand corner.
Assessment Reporting
Basic Assessment Reports
- List of Assessments
- List of Questions
- Assessment Questions Report
Other Reports relevant to Assessments
- Last Assessment Attempt Questions and Answers Report
- Did my Employee Complete a Specific Assessment Report
- Training History Report
Further Assessment Considerations/Information
Using an assessment in a pathway step
- Importance of the assessment name and the intervention name to be the same
- Settings available on the pathway step
Types of Assessments things to consider / more information
- 360 Questionnaire
- Clicker Assessment
- Diminishing Questions
Employee Assessments
- View a list of Assessments attempted/completed by an employee and their Scores
- Viewing a list of all attempts on an Assessment by an Employee
- Viewing the transcript of an attempt on an Assessment by an Employee
- Reset all attempts for a specific Assessment of an employee