The case is the multi-select, multiple choice question, where a learner may select x number of correct answers from y number of options.

The client particularly did not want to divulge how many correct answers there were, so out of 8 options, the learner could choose anything up to 8 answers. The client’s point of view was that telling the learner how many answers were correct was too much of a clue, and that part of what they were seeking to test was the learners’ ability to confidently distinguish the true statements from the false, rather than just guess the most plausible.

However, this can have large implications.

If we say “Choose the correct 2 options from the 8 options listed”, the learner must always choose 2 options. This means the total possible number of responses are:

8!/2!⋅(8−2)!​=28

So there are 28 ways to answer this question.

But if we don’t tell the learner there are 2 correct options they could choose 1, 2, or anything up to all 8. The total number possible responses is the total of the possible combinations. So…

8+28+56+70+56+28+8+1=255

Depending on how the marking rubric is stored, this could make quite a difference. A flat table of possible responses would require 255 rows.

Where it gets even more complicated is when we want to give partial scores for partially correct answers. Lets say the learner gets 2 marks for each correct selection. 4 marks in total are possible. But what if they choose the 2 correct options and a third incorrect one? What if they choose six? It is not obvious how such responses would be scored.

I posed this question to Co-Pilot and it simply recommended specifying the correct number of options to minimize these difficulties. So it seems there are no easy answers here!

I typed this into Co-Pilot:

Given an average rating out of 5 and a count of the number of ratings, and assuming every good review is 5 and every bad review is 1, create a function in excel to calculate the number of bad reviews

Co-pilot’s response:

Let’s assume:
A1 contains the average rating.
B1 contains the total number of ratings.
The formula to calculate the number of bad reviews (where each bad review is rated as 1) is:
= (5*B1 – A1*B1) / 4

Here’s the breakdown of the formula:
5*B1 calculates the total points if all reviews were good (rated 5).
A1*B1 calculates the total points actually received.
The difference (5*B1 – A1*B1) gives the total points lost due to bad reviews.
Dividing by 4 gives the number of bad reviews because each bad review contributes 4 points less than a good review (5-1=4).

Pretty clever!