we are happy to present you our currently implemented grading logic in a bit more detail in this post. This logic applies to the points that participants collect when answering during live sessions, learning elements or micro-sessions both on a session and course level. To track performance across courses, a concept including experience points, user levels and achievements is currently under development and will be published in a separate post later on.
The presented grading approach assumes that all considered questions have a defined correct solution (or possibly multiple solutions). During live sessions where questions without sample solutions were used, the participants are awarded 10 points for taking part in each poll. This number and other fixed point amounts are still subject to change until the stable release of KlickerUZH v3.0.
Please also note that the grading approach differs slightly between live sessions and learning elements / micro-sessions and that some question types might not be supported at the moment, depending on the session type.
The grading logic will first be described on the example of learning elements and micro-sessions, which do not include any live elements. The section on live sessions below will then describe the supplements that were implemented for the synchronous setting.
The point multipliers, which can be specified both on a question and learning element / micro-session level, are combined during the creation of the session and applied to the total awarded points (including basic points and potential bonuses). If no multipliers are specified, this factor defaults to 1.
Multipliers can be used to weigh questions according to their difficulty and reward participation in certain polls with more points (e.g. poll on exam level during the last lecture of the semester vs. an introductory pool in the first lecture).
The same multiplier concept also applies to live sessions.
Single choice questions have a single correct answer by definition, wherefore participants will receive 10 points, multiplied with any potential multipliers, if they answer correctly.
For multiple choice questions with n answer options, 0 to n answers can be correct. Each of the answers is considered separately during grading, and every mismatch between the correct solution and the participant’s input is counted as one mistake. The awarded number of points then depends on the percentage of correctly answered questions:
The participant will therefore receive full points for a completely correct response and partial points until less than 50% of the answers are correct. In the shown example, there were 7 potentially correct answer options.
The grading logic for KPRIM questions is very similar to the presented implementation for multiple choice questions as KPRIM questions can be seen as a special case of multiple choice questions with exactly four answer options and a unified grading scheme. Given the fixed number of questions, participants will receive full points for a completely correct answer, half the maximum points for a single mistake and no points otherwise.
For numerical questions, you will be able to specify a number of solution ranges. If the participant’s entered input is within one of these ranges, it is considered to be correct and will be awarded full points and no points otherwise (as for single choice questions).
For free text questions, you can currently specify a selection of correct responses. The implemented simple grading approach compares the participant’s input to this sample solution, ignoring capitalization of letters and whitespaces. Again, full points are awarded for correct answers and no points otherwise (as for single choice questions).
During synchronous live sessions, every participant will receive 10 points for each submitted answer, independent of its correctness. If the submitted answer is correct, 5 additional points will be added. For partially correct answers, these 5 points are multiplied with a factor described in the sections above. If no sample solution is defined, students will not receive any time-dependent bonus, but just the default 10 points (no multiplication with multiplier).
To incentivize fast and correct answers during live sessions, additional bonus points are awarded. Starting time with the first correct answer, players will receive up to 45 points, depending on the duration between their answer and the first one. The current slope to zero points is implemented with a duration of 20 seconds.
The corresponding resulting point curves for correct and wrong answers during live sessions, when starting time with the first correct response, are shown in the plot below (not considering multipliers). As already mentioned, the absolute numbers might still be subject to change.