Dear Users,
we are happy to present you with our currently implemented grading logic in a bit more detail in this post. This logic applies to the points that participants collect when answering during live quizzes, practice quizze, or microlearning both on a quiz and course level.
The presented grading approach assumes that all considered questions have a defined correct solution (or possibly multiple solutions). During live sessions where questions without sample solutions were used, the participants are awarded 10 points for taking part in each poll. This number and other fixed point amounts are still subject to change until the stable release of KlickerUZH v3.0.
Please also note that the grading approach differs slightly between live quizzes and practice quizzes / microlearning and that some question types might not be supported at the moment, depending on the activity type.
To track performance across courses, a concept including experience points, user levels and achievements is currently under development and will be published in a separate post later on.
Grading by Question Type
The grading logic will first be described on the example of practice quizzes and micro-sessions, which do not include any live quiz elements. The section on live quizzes below will then describe the additions that were implemented for the synchronous setting.
Multiplier
The point multipliers, which can be specified both on a question and learning element / micro-session level, are combined during the creation of the session and applied to the total awarded points (including basic points and potential bonuses). If no multipliers are specified, this factor defaults to 1.
Multipliers can be used to weigh questions according to their difficulty and reward participation in certain quizzes with more points (e.g. quiz on exam level during the last lecture of the semester vs. an introductory quiz in the first lecture).
The same multiplier concept also applies to live sessions.
Grading Single Choice Questions
Single choice questions have a single correct answer by definition, wherefore participants will receive 10 points, multiplied with any potential multipliers, if they answer correctly.
Grading Multiple Choice Questions
For multiple choice questions with n answer options, 0 to n answers can be correct. Each of the answers is considered separately during grading, and every mismatch between the correct solution and the participant’s input is counted as one mistake. The awarded number of points then depends on the percentage of correctly answered questions:
The participant will therefore receive full points for a completely correct response and partial points until less than 50% of the answers are correct. In the shown example, there were 7 potentially correct answer options.
Grading KPRIM Questions
The grading logic for KPRIM questions is very similar to the presented implementation for multiple choice questions as KPRIM questions can be seen as a special case of multiple choice questions with exactly four answer options and a unified grading scheme. Given the fixed number of questions, participants will receive full points for a completely correct answer, half the maximum points for a single mistake and no points otherwise.
Grading Numerical Questions
For numerical questions, you will be able to specify a number of solution ranges. If the participant’s entered input is within one of these ranges, it is considered to be correct and will be awarded full points and no points otherwise (as for single choice questions).
Grading Free Text Questions
For free text questions, you can currently specify a selection of correct responses. The implemented simple grading approach compares the participant’s input to this sample solution, ignoring capitalization of letters and whitespaces. Again, full points are awarded for correct answers and no points otherwise (as for single choice questions).
Grading in Live Quizzes
During synchronous live quizzes, every participant will receive 10 points for each submitted answer, independent of its correctness. If the submitted answer is correct, 5 additional points will be added. For partially correct answers, these 5 points are multiplied with a factor described in the sections above. If no sample solution is defined, students will not receive any time-dependent bonus, but just the default 10 points (no multiplication with multiplier).
To incentivize fast and correct answers during live sessions, additional bonus points are awarded. Starting time with the first correct answer, players will receive up to 45 points, depending on the duration between their answer and the first one. The current slope to zero points is implemented with a duration of 20 seconds.
The corresponding resulting point curves for correct and wrong answers during live sessions, when starting time with the first correct response, are shown in the plot below (not considering multipliers). As already mentioned, the absolute numbers might still be subject to change.
