Note
EdX offers full support for this problem type.
In checkbox problems, learners select one or more options from a list of possible answers. To answer the problem correctly, a learner must select all of the options that are correct answers, and none of the options that are incorrect. The course team must set up each checkbox problem to have at least one correct answer.
Note
Make sure that all of the answer choices are unambiguous, and avoid trick questions. Checkbox problems with ambiguity can be frustrating to learners, especially if the problems have a limited number of attempts.
For the checkbox problems in your course, you can use edX Insights to review aggregated learner performance data and examine submitted answers. For more information, see Using edX Insights.
You can create checkbox problems in the simple editor or in the advanced editor. You can set up a problem in the simple editor, and then switch to the advanced editor to add more configuration options in XML. However, you cannot switch back to the simple editor from the advanced editor. Therefore, you might want to format the problem as completely as possible before you begin to use the advanced editor.
To use the simple editor to create a checkbox problem, follow these steps.
From the list of Common Problem Types, select Checkboxes.
From the list of Common Problems with Hints and Feedback, select Checkboxes with Hints and Feedback. For more information, see Use Feedback in a Checkbox Problem.
Studio adds the checkbox problem to the unit.
>>question<<
).
This text identifies the question for screen readers, reports, and Insights.[explanation]
appears before
and after the explanation text.For the example problem illustrated above, the following text displays in the problem component.
Learning about the benefits of preventative healthcare can be particularly
difficult. >>Check all of the reasons below why this may be the case.<<
[x] A large amount of time passes between undertaking a preventative measure and seeing the result.
[ ] Non-immunized people will always fall sick.
[x] If others are immunized, fewer people will fall sick regardless of a particular individual's choice to get immunized or not.
[x] Trust in healthcare professionals and government officials is fragile.
To use the advanced editor to edit a checkbox problem, follow these steps.
<problem>
<p>Learning about the benefits of preventative healthcare can be particularly difficult. Check all of the reasons below why this may be the case.</p>
<choiceresponse>
<checkboxgroup label="Check all of the reasons below why this may be the case">
<choice correct="true"><text>A large amount of time passes between undertaking a preventative measure and seeing the result.</text></choice>
<choice correct="false"><text>Non-immunized people will always fall sick.</text></choice>
<choice correct="true"><text>If others are immunized, fewer people will fall sick regardless of a particular individual's choice to get immunized or not.</text></choice>
<choice correct="true"><text>Trust in healthcare professionals and government officials is fragile.</text></choice>
</checkboxgroup>
</choiceresponse>
<solution>
<div class="detailed-solution">
<p>Explanation</p>
<p>People who are not immunized against a disease may still not fall sick from the disease. If someone is trying to learn whether or not preventative measures against the disease have any impact, he or she may see these people and conclude, since they have remained healthy despite not being immunized, that immunizations have no effect. Consequently, he or she would tend to believe that immunization (or other preventative measures) have fewer benefits than they actually do.</p>
</div>
</solution>
</problem>
You can add feedback in a checkbox problem using the simple editor or the advanced editor. For an overview of feedback in problems, see Adding Feedback and Hints to a Problem.
In checkbox problems, you can provide feedback for each option that a learner can select, with distinct feedback depending on whether or not the learner selects that option. This means that there are four possible types of feedback.
You can configure the checkbox problem to provide compound feedback. Compound feedback is feedback given for a specific combination of options. For example, if you have three possible option in the problem, you can define specific feedback for when a learner selects each combination of possible options.
For problems with more than three options, providing specific feedback for each combination can become difficult. For such problems, you can define compound feedback for more likely combinations of option or for combinations of option that reflect common learner misunderstandings. If you do not define feedback for a combination that a learner selects, the learner receives feedback for the individual selections.
You can configure individual option and compound feedback in the simple editor. When you create a new checkbox problem, select the template Checkboxes with Hints and Feedback. This template has example feedback syntax that you can replace.
In the simple editor, you configure individual option feedback with the following syntax.
[x] answer {{ selected: Feedback when learner selects this option. },
{unselected: Feedback when the learner does not select this option.}}
Note
You can use S
for selected
and U
for unselected.
For example, the following problem has feedback for each option.
>>Which of the following is an example of a fruit? Check all that apply.<<
[x] apple {{ selected: You are correct that an apple is a fruit because it
is the fertilized ovary that comes from an apple tree and contains seeds. },
{ unselected: Remember that an apple is also a fruit.}}
[x] pumpkin {{ selected: You are correct that a pumpkin is a fruit because it
is the fertilized ovary of a squash plant and contains seeds.}, { unselected:
Remember that a pumpkin is also a fruit.}}
[ ] potato {{ U: You are correct that a potato is a vegetable because it is
an edible part of a plant in tuber form.}, { S: A potato is a vegetable, not
a fruit, because it does not come from the flower on a plant or tree and does
not contain seeds.}}
[x] tomato {{ S: You are correct that a tomato is a fruit because it is the
fertilized ovary of a tomato plant and contains seeds. }, { U: Many people
mistakenly think a tomato is a vegetable. However, because a tomato is the
fertilized ovary of a tomato plant and contains seeds it is classified as a
fruit.}}
In the simple editor, you configure compound feedback after the possible options, with the following syntax.
{{ ((Answer Combination)) Feedback when learner selects this combination of
options.}}
For example, the following compound feedback is used when learners select options A, B, and D or A, B, C, and D.
{{ ((A B D)) An apple, pumpkin, and tomato are all fruits as they are all the
fertilized ovaries of a plant and contain seeds. }}
{{ ((A B C D)) You are correct that an apple, pumpkin, and tomato are all
fruits as they are all the fertilized ovaries of a plant and contain seeds.
However, a potato is not a fruit as it is an edible part of a plant in tuber
form and is classified as a vegetable. }}
You can configure individual option and compound feedback in the Advanced Editor.
In the advanced editor, you configure individual option feedback with the following syntax.
<choice correct="true">Choice label
<choicehint selected="true">
Feedback for when learner selects this answer.
</choicehint>
<choicehint selected="false">
Feedback for when learner does not select this answer.
</choicehint>
</choice>
For example, the following problem has feedback for each option.
<choiceresponse>
<checkboxgroup label="Which of the following is an example
of a fruit? Check all that apply.">
<choice correct="true">apple
<choicehint selected="true">You are correct that an apple is a fruit
because it is the fertilized ovary that comes from an apple tree and
contains seeds.
</choicehint>
<choicehint selected="false">Remember that an apple is also a
fruit.
</choicehint>
</choice>
<choice correct="true">pumpkin
<choicehint selected="true">You are correct that a pumpkin is a fruit
because it is the fertilized ovary of a squash plant and contains
seeds.
</choicehint>
<choicehint selected="false">Remember that a pumpkin is also a
fruit.
</choicehint>
</choice>
<choice correct="false">potato
<choicehint selected="true">A potato is a vegetable, not a fruit,
because it does not come from the flower on a plant or tree and does
not contain seeds.
</choicehint>
<choicehint selected="false">You are correct that a potato is
classified as a vegetable because it is an edible part of a plant in
tuber form.
</choicehint>
</choice>
<choice correct="true">tomato
<choicehint selected="true">You are correct that a tomato is
classified as a fruit because it is the fertilized ovary of a tomato
plant and contains seeds.
</choicehint>
<choicehint selected="false">Many people mistakenly think a tomato is
a vegetable. However, because a tomato is the fertilized ovary of a
tomato plant and contains seeds it is classified as a fruit.
</choicehint>
</choice>
</checkboxgroup>
</choiceresponse>
In the advanced editor, you define compound feedback in the <compoundhint>
element within the <checkboxgroup>
element.
<compoundhint value="Answer Combination">
Feedback when learner selects this combination of answers.
</compoundhint>}}
For example, the following compound feedback is used when learners select options A, B, and D or A, B, C, and D.
<compoundhint value="A B D">An apple, pumpkin, and tomato are all fruits as
they all are fertilized ovaries of a plant and contain seeds.
</compoundhint>
<compoundhint value="A B C D">You are correct that an apple, pumpkin, and
tomato are all fruits as they all are fertilized ovaries of a plant and
contain seeds. However, a potato is not a fruit as it is an edible part of
a plant in tuber form and is classified as a vegetable.
</compoundhint>
You can add hints to a checkbox problem, using the simple editor or the advanced editor. For an overview of hints in problems, see Adding Feedback and Hints to a Problem.
In the simple editor, you configure hints with the following syntax.
||Hint 1||
||Hint 2||
||Hint n||
Note
You can configure any number of hints. The learner views one hint at a time and views the next one by selecting Hint again.
For example, the following problem has two hints.
||A fruit is the fertilized ovary from a flower.||
||A fruit contains seeds of the plant.||
In the advanced editor, you configure each hint in the <hint>
element
within the <demandhint>
element.
<demandhint>
<hint>Hint 1</hint>
<hint>Hint 2</hint>
<hint>Hint 3</hint>
</demandhint>
For example, the following XML shows two hints.
<demandhint>
<hint>A fruit is the fertilized ovary from a flower.</hint>
<hint>A fruit contains seeds of the plant.</hint>
</demandhint>
You can configure a checkbox problem to award partial credit to learners who submit an answer that is partly correct. You must use the advanced editor to configure partial credit.
For an overview of partial credit in problems, see Awarding Partial Credit for a Problem.
In the following example, the learner selected two of the three correct choices, and did not select any incorrect choices. The learner therefore had four out of five correct answers. Because the course team set this problem up to award partial credit for every correct answer selected and every incorrect answer left unselected (known as every decision counts), the learner earned 80% of the points for this problem.
There are two ways to award partial credit in a checkbox problem.
You can configure a checkbox problem so that the learner’s response for every option is evaluated and scored. This is known as “every decision counts” (EDC).
With EDC, for each option the learner gets wrong, either by not selecting a correct option or selecting an incorrect option, the learner’s score is reduced by 1/n, where “n” is the number of options.
For example, if there are four options, each one is worth 25% of the total score. If a learner’s response is wrong for one option, she receives 75% of the points for the problem.
The following table describes the learner’s score for different submissions for EDC problems with a variety of correct answer options.
Learner’s Selections | Correct Answers | Incorrect Answers | Score |
---|---|---|---|
A, B, C | A, B, D | C | 75% |
A | A, C, D | B | 75% |
A, C | A, D | B, C | 50% |
C, D | A, B, C, D | 0% |
To configure an EDC checkbox problem, you add the partial_credit="EDC"
attribute to the <choiceresponse>
element in the problem XML.
For example, the following XML shows the checkbox problem template after it is updated to provide partial credit.
<choiceresponse partial_credit="EDC">
<checkboxgroup label="Which of the following is a fruit? Check all that apply.">
<choice correct="true">apple</choice>
<choice correct="true">pumpkin</choice>
<choice correct="false">potato</choice>
<choice correct="true">tomato</choice>
</checkboxgroup>
</choiceresponse>
You can configure a checkbox problem so that for every option that a learner gets wrong, either by not selecting a correct option or by selecting an incorrect option, half of the remaining points are deducted from the learner’s score. This is known as scoring by “halves”.
Note
By design, partial credit by halves requires the number of answer options to be more than twice the number of incorrect answers. In addition, partial credit is not given for more than two wrong answers, regardless of the total number of answer options. In other words, two wrong answers is scored at 25% only if there are at least 5 answer options. Three or more wrong answers is always scored at 0%, regardless of the number of total answer options.
Partial credit using the halves method is calculated as follows.
The following tables illustrate partial credit score using the halves method, for problems with an increasing number of total answer options.
Number of Incorrect Answers | Number of Answer Options | Credit Given (%) |
---|---|---|
0 | 2 | 100 |
1 | 2 | 0 |
2 | 2 | 0 |
Number of Incorrect Answers | Number of Answer Options | Credit Given (%) |
---|---|---|
0 | 3 | 100 |
1 | 3 | 0 |
2 | 3 | 0 |
3 | 3 | 0 |
Number of Incorrect Answers | Number of Answer Options | Credit Given (%) |
---|---|---|
0 | 4 | 100 |
1 | 4 | 50 |
2 | 4 | 0 |
3 | 4 | 0 |
4 | 4 | 0 |
Number of Incorrect Answers | Number of Answer Options | Credit Given (%) |
---|---|---|
0 | 5 | 100 |
1 | 5 | 50 |
2 | 5 | 25 |
3 | 5 | 0 |
4 | 5 | 0 |
5 | 5 | 0 |
Number of Incorrect Answers | Number of Answer Options | Credit Given (%) |
---|---|---|
0 | 7 | 100 |
1 | 7 | 50 |
2 | 7 | 25 |
3 | 7 | 0 |
4 | 7 | 0 |
5 | 7 | 0 |
To configure a halves checkbox problem, you add the
partial_credit="halves"
attribute to the <choiceresponse>
element in
the problem XML.
For example, the following XML shows the checkbox problem template updated to provide partial credit.
<choiceresponse partial_credit="halves">
<checkboxgroup label="Which of the following is a fruit? Check all that apply.">
<choice correct="true">apple</choice>
<choice correct="true">pumpkin</choice>
<choice correct="false">potato</choice>
<choice correct="true">tomato</choice>
</checkboxgroup>
</choiceresponse>
<problem>
<legend>Question text</legend>
<choiceresponse>
<checkboxgroup label="label text">
<choice correct="false">
Answer option A (incorrect)
<choicehint selected="true">
Feedback for when learner selects this answer.
</choicehint>
<choicehint selected="false">
Feedback for when learner does not select this answer.
</choicehint>
</choice>
<choice correct="true">
Answer option B (correct)
<choicehint selected="true">
Feedback for when learner selects this answer.
</choicehint>
<choicehint selected="false">
Feedback for when learner does not select this answer.
</choicehint>
</choice>
<choice correct="false">
Answer option C (correct)
<choicehint selected="true">
Feedback for when learner selects this answer.
</choicehint>
<choicehint selected="false">
Feedback for when learner does not select this answer.
</choicehint>
</choice>
<compoundhint value="A B">
Feedback when answers A and B are selected.
</compoundhint>
<compoundhint value="A C">
Feedback when answers A and C are selected.
</compoundhint>
</checkboxgroup>
</choiceresponse>
<demandhint>
<hint>Hint 1</hint>
<hint>Hint 2</hint>
<hint>Hint 3</hint>
</demandhint>
<solution>
<div class="detailed-solution">
<p>Solution or Explanation Heading</p>
<p>Solution or explanation text</p>
</div>
</solution>
</problem>
<choiceresponse>
(required): Specifies that the problem contains options
for learners to choose from.<checkboxgroup>
(required): Specifies that the problem is a checkbox
problem.<compoundhint>
(optional): Specifies feedback for a specific combination
of answers.<choice>
(required): Designates an answer option.<demandhint>
(optional): Specifies hints for the learner.Tag: <choiceresponse>
Specifies that the problem contains options for learners to choose from.
Attributes
Attribute Description partial_credit (optional) Specifies the type of partial credit given. EDC
orhalves
.Children
<checkboxgroup>
Tag: <checkboxgroup>
Specifies that the problem is a checkbox problem.
Attributes
Attribute Description label (required) Specifies the name of the response field. Children
<choice>
<compoundhint>
Tag: <choice>
Designates an answer option.
Attributes
Attribute Description true (at least one required) Indicates a correct answer. For checkbox problems, one or more <choice>
tags can contain a correct answer.false (at least one required) Indicates an incorrect answer. Children
<choicehint>
Tag: <choicehint>
Specifies a hint for the answer option.
Attributes
Attribute Description selected (required) true
orfalse
. Indicates if the hint is given when the answer option is selected, or when it is not selected.Children
(none)
Tag: <compoundhint>
Designates feedback for a specific combination of answers.
Attributes
Attribute Description value (at least one required) Indicates which combination of selected answers triggers this feedback. Children
(none)
Tag: <demandhint>
Specifies hints available to the learner.
Children
<hint>
Tag: <hint>
Specifies a hint available to the learner.
Children
(none)