8.5. Checkbox Problem

Note

EdX offers full support for this problem type.

8.5.1. Overview

In checkbox problems, learners select one or more options from a list of possible answers. To answer the problem correctly, a learner must select all of the options that are correct answers, and none of the options that are incorrect. The course team must set up each checkbox problem to have at least one correct answer.

A checkbox problem with four options, two of which are correct.

Note

Make sure that all of the answer choices are unambiguous, and avoid trick questions. Checkbox problems with ambiguity can be frustrating to learners, especially if the problems have a limited number of attempts.

8.5.2. Analyzing Performance on Checkbox Problems

For the checkbox problems in your course, you can use edX Insights to review aggregated learner performance data and examine submitted answers. For more information, see Using edX Insights.

8.5.3. Creating a Checkbox Problem

You can create checkbox problems in the simple editor or in the advanced editor. You can set up a problem in the simple editor, and then switch to the advanced editor to add more configuration options in XML. However, you cannot switch back to the simple editor from the advanced editor. Therefore, you might want to format the problem as completely as possible before you begin to use the advanced editor.

8.5.3.1. Use the Simple Editor to Create a Checkbox Problem

To use the simple editor to create a checkbox problem, follow these steps.

  1. In the unit where you want to create the problem, under Add New Component select Problem.
  2. Select one of the two checkbox problem templates.
  • From the list of Common Problem Types, select Checkboxes.

  • From the list of Common Problems with Hints and Feedback, select Checkboxes with Hints and Feedback. For more information, see Use Feedback in a Checkbox Problem.

    Studio adds the checkbox problem to the unit.

  1. Select Edit. The simple editor opens.
  2. Replace the sample problem text with your own text.
  3. Determine the text that describes the question you want learners to answer, and surround that text with two pairs of angle brackets (>>question<<). This text identifies the question for screen readers, reports, and Insights.
  4. Edit the text to place each answer option on a separate line.
  5. Select the set of answer options, and then select Checkboxes from the toolbar. A pair of brackets appears next to each answer choice.
  6. To identify each correct answer, add an x between the brackets for that option.
  7. To provide an explanation, select the explanation text and then select Explanation from the toolbar. [explanation] appears before and after the explanation text.
  8. Select Settings and provide an identifying Display Name for the problem.
  9. Define additional settings for the problem. For more information, see Problem Settings.
  10. Select Save.

For the example problem illustrated above, the following text displays in the problem component.

Learning about the benefits of preventative healthcare can be particularly
difficult. >>Check all of the reasons below why this may be the case.<<

[x] A large amount of time passes between undertaking a preventative measure and seeing the result.
[ ] Non-immunized people will always fall sick.
[x] If others are immunized, fewer people will fall sick regardless of a particular individual's choice to get immunized or not.
[x] Trust in healthcare professionals and government officials is fragile.

8.5.3.2. Use the Advanced Editor to Edit a Checkbox Problem

To use the advanced editor to edit a checkbox problem, follow these steps.

  1. Follow the steps for creating the problem in the simple editor.
  2. Select Advanced Editor, and then edit the XML to add the tags and attributes you want. An example follows.
<problem>
<p>Learning about the benefits of preventative healthcare can be particularly difficult. Check all of the reasons below why this may be the case.</p>

<choiceresponse>
  <checkboxgroup label="Check all of the reasons below why this may be the case">
    <choice correct="true"><text>A large amount of time passes between undertaking a preventative measure and seeing the result.</text></choice>
    <choice correct="false"><text>Non-immunized people will always fall sick.</text></choice>
    <choice correct="true"><text>If others are immunized, fewer people will fall sick regardless of a particular individual's choice to get immunized or not.</text></choice>
    <choice correct="true"><text>Trust in healthcare professionals and government officials is fragile.</text></choice>
  </checkboxgroup>
</choiceresponse>

<solution>
  <div class="detailed-solution">
    <p>Explanation</p>
    <p>People who are not immunized against a disease may still not fall sick from the disease. If someone is trying to learn whether or not preventative measures against the disease have any impact, he or she may see these people and conclude, since they have remained healthy despite not being immunized, that immunizations have no effect. Consequently, he or she would tend to believe that immunization (or other preventative measures) have fewer benefits than they actually do.</p>
 </div>
</solution>
</problem>

8.5.4. Using Feedback in a Checkbox Problem

You can add feedback in a checkbox problem using the simple editor or the advanced editor. For an overview of feedback in problems, see Adding Feedback and Hints to a Problem.

In checkbox problems, you can provide feedback for each option that a learner can select, with distinct feedback depending on whether or not the learner selects that option. This means that there are four possible types of feedback.

  • The learner selects a correct option. This type of feedback should indicate why the option is correct.
  • The learner does not select a correct option. This type of feedback should indicate that the learner missed checking this option and why it is correct.
  • The learner selects an incorrect option. This type of feedback should indicate that the learner incorrectly checked this option and why it is incorrect
  • The learner does not select an incorrect option. This type of feedback should reinforce why the learner correctly left this option unselected.

8.5.4.1. Compound Feedback

You can configure the checkbox problem to provide compound feedback. Compound feedback is feedback given for a specific combination of options. For example, if you have three possible option in the problem, you can define specific feedback for when a learner selects each combination of possible options.

  • A
  • B
  • C
  • A, B
  • B, C
  • A, C
  • A, B, C

For problems with more than three options, providing specific feedback for each combination can become difficult. For such problems, you can define compound feedback for more likely combinations of option or for combinations of option that reflect common learner misunderstandings. If you do not define feedback for a combination that a learner selects, the learner receives feedback for the individual selections.

8.5.4.2. Configure Feedback in the Simple Editor

You can configure individual option and compound feedback in the simple editor. When you create a new checkbox problem, select the template Checkboxes with Hints and Feedback. This template has example feedback syntax that you can replace.

8.5.4.2.1. Configure Feedback for Individual Options

In the simple editor, you configure individual option feedback with the following syntax.

[x] answer  {{ selected: Feedback when learner selects this option. },
{unselected: Feedback when the learner does not select this option.}}

Note

You can use S for selected and U for unselected.

For example, the following problem has feedback for each option.

>>Which of the following is an example of a fruit? Check all that apply.<<

[x] apple  {{ selected: You are correct that an apple is a fruit because it
is the fertilized ovary that comes from an apple tree and contains seeds. },
{ unselected: Remember that an apple is also a fruit.}}

[x] pumpkin {{ selected: You are correct that a pumpkin is a fruit because it
is the fertilized ovary of a squash plant and contains seeds.}, { unselected:
Remember that a pumpkin is also a fruit.}}

[ ] potato   {{ U: You are correct that a potato is a vegetable because it is
an edible part of a plant in tuber form.}, { S: A potato is a vegetable, not
a fruit, because it does not come from the flower on a plant or tree and does
not contain seeds.}}

[x] tomato  {{ S: You are correct that a tomato is a fruit because it is the
fertilized ovary of a tomato plant and contains seeds. }, { U: Many people
mistakenly think a tomato is a vegetable. However, because a tomato is the
fertilized ovary of a tomato plant and contains seeds it is classified as a
fruit.}}

8.5.4.2.2. Configure Compound Feedback

In the simple editor, you configure compound feedback after the possible options, with the following syntax.

{{ ((Answer Combination)) Feedback when learner selects this combination of
options.}}

For example, the following compound feedback is used when learners select options A, B, and D or A, B, C, and D.

{{ ((A B D)) An apple, pumpkin, and tomato are all fruits as they are all the
fertilized ovaries of a plant and contain seeds. }}

{{ ((A B C D)) You are correct that an apple, pumpkin, and tomato are all
fruits as they are all the fertilized ovaries of a plant and contain seeds.
However, a potato is not a fruit as it is an edible part of a plant in tuber
form and is classified as a vegetable.  }}

8.5.4.3. Configure Feedback in the Advanced Editor

You can configure individual option and compound feedback in the Advanced Editor.

8.5.4.3.1. Configure Individual Option Feedback

In the advanced editor, you configure individual option feedback with the following syntax.

<choice correct="true">Choice label
  <choicehint selected="true">
    Feedback for when learner selects this answer.
  </choicehint>
  <choicehint selected="false">
    Feedback for when learner does not select this answer.
  </choicehint>
</choice>

For example, the following problem has feedback for each option.

<choiceresponse>
  <checkboxgroup label="Which of the following is an example
  of a fruit? Check all that apply.">
    <choice correct="true">apple
      <choicehint selected="true">You are correct that an apple is a fruit
        because it is the fertilized ovary that comes from an apple tree and
        contains seeds.
      </choicehint>
      <choicehint selected="false">Remember that an apple is also a
        fruit.
      </choicehint>
    </choice>
    <choice correct="true">pumpkin
      <choicehint selected="true">You are correct that a pumpkin is a fruit
        because it is the fertilized ovary of a squash plant and contains
        seeds.
      </choicehint>
      <choicehint selected="false">Remember that a pumpkin is also a
        fruit.
      </choicehint>
    </choice>
    <choice correct="false">potato
      <choicehint selected="true">A potato is a vegetable, not a fruit,
        because it does not come from the flower on a plant or tree and does
        not contain seeds.
      </choicehint>
      <choicehint selected="false">You are correct that a potato is
        classified as a vegetable because it is an edible part of a plant in
        tuber form.
      </choicehint>
    </choice>
    <choice correct="true">tomato
      <choicehint selected="true">You are correct that a tomato is
        classified as a fruit because it is the fertilized ovary of a tomato
        plant and contains seeds.
      </choicehint>
      <choicehint selected="false">Many people mistakenly think a tomato is
        a vegetable. However, because a tomato is the fertilized ovary of a
        tomato plant and contains seeds it is classified as a fruit.
      </choicehint>
    </choice>
  </checkboxgroup>
</choiceresponse>

8.5.4.3.2. Configure Compound Feedback

In the advanced editor, you define compound feedback in the <compoundhint> element within the <checkboxgroup> element.

<compoundhint value="Answer Combination">
  Feedback when learner selects this combination of answers.
</compoundhint>}}

For example, the following compound feedback is used when learners select options A, B, and D or A, B, C, and D.

<compoundhint value="A B D">An apple, pumpkin, and tomato are all fruits as
  they all are fertilized ovaries of a plant and contain seeds.
</compoundhint>
<compoundhint value="A B C D">You are correct that an apple, pumpkin, and
  tomato are all fruits as they all are fertilized ovaries of a plant and
  contain seeds. However, a potato is not a fruit as it is an edible part of
  a plant in tuber form and is classified as a vegetable.
</compoundhint>

8.5.5. Using Hints in a Checkbox Problem

You can add hints to a checkbox problem, using the simple editor or the advanced editor. For an overview of hints in problems, see Adding Feedback and Hints to a Problem.

8.5.5.1. Configure Hints in the Simple Editor

In the simple editor, you configure hints with the following syntax.

||Hint 1||
||Hint 2||
||Hint n||

Note

You can configure any number of hints. The learner views one hint at a time and views the next one by selecting Hint again.

For example, the following problem has two hints.

||A fruit is the fertilized ovary from a flower.||
||A fruit contains seeds of the plant.||

8.5.5.2. Configure Hints in the Advanced Editor

In the advanced editor, you configure each hint in the <hint> element within the <demandhint> element.

<demandhint>
  <hint>Hint 1</hint>
  <hint>Hint 2</hint>
  <hint>Hint 3</hint>
</demandhint>

For example, the following XML shows two hints.

<demandhint>
  <hint>A fruit is the fertilized ovary from a flower.</hint>
  <hint>A fruit contains seeds of the plant.</hint>
</demandhint>

8.5.6. Awarding Partial Credit in a Checkbox Problem

You can configure a checkbox problem to award partial credit to learners who submit an answer that is partly correct. You must use the advanced editor to configure partial credit.

For an overview of partial credit in problems, see Awarding Partial Credit for a Problem.

In the following example, the learner selected two of the three correct choices, and did not select any incorrect choices. The learner therefore had four out of five correct answers. Because the course team set this problem up to award partial credit for every correct answer selected and every incorrect answer left unselected (known as every decision counts), the learner earned 80% of the points for this problem.

A checkbox choice problem with partial credit for two out of three answers.

There are two ways to award partial credit in a checkbox problem.

8.5.6.1. Using the Every Decision Counts Style

You can configure a checkbox problem so that the learner’s response for every option is evaluated and scored. This is known as “every decision counts” (EDC).

With EDC, for each option the learner gets wrong, either by not selecting a correct option or selecting an incorrect option, the learner’s score is reduced by 1/n, where “n” is the number of options.

For example, if there are four options, each one is worth 25% of the total score. If a learner’s response is wrong for one option, she receives 75% of the points for the problem.

The following table describes the learner’s score for different submissions for EDC problems with a variety of correct answer options.

Learner’s Selections Correct Answers Incorrect Answers Score
A, B, C A, B, D C 75%
A A, C, D B 75%
A, C A, D B, C 50%
C, D   A, B, C, D 0%

8.5.6.1.1. Configure an EDC Checkbox Problem

To configure an EDC checkbox problem, you add the partial_credit="EDC" attribute to the <choiceresponse> element in the problem XML.

For example, the following XML shows the checkbox problem template after it is updated to provide partial credit.

<choiceresponse partial_credit="EDC">
  <checkboxgroup label="Which of the following is a fruit? Check all that apply.">
    <choice correct="true">apple</choice>
    <choice correct="true">pumpkin</choice>
    <choice correct="false">potato</choice>
    <choice correct="true">tomato</choice>
  </checkboxgroup>
</choiceresponse>

8.5.6.2. Using the Halves Style

You can configure a checkbox problem so that for every option that a learner gets wrong, either by not selecting a correct option or by selecting an incorrect option, half of the remaining points are deducted from the learner’s score. This is known as scoring by “halves”.

Note

By design, partial credit by halves requires the number of answer options to be more than twice the number of incorrect answers. In addition, partial credit is not given for more than two wrong answers, regardless of the total number of answer options. In other words, two wrong answers is scored at 25% only if there are at least 5 answer options. Three or more wrong answers is always scored at 0%, regardless of the number of total answer options.

Partial credit using the halves method is calculated as follows.

  • If a learner makes no errors, she receives full credit for the problem.
  • If a learner makes one error, she receives 50% of the possible points, as long as there are three or more choices in the problem. If a learner makes one error and there are only two choices in the problem, no credit is given.
  • If a learner makes two errors, she receives 25% of the possible points, as long as there are five or more choices in the problem. If a learner makes two errors and there are only three choices or four choices in the problem, no credit is given.
  • If a learner makes three errors, she receives no credit for the problem, regardless of how many answer options there are.

The following tables illustrate partial credit score using the halves method, for problems with an increasing number of total answer options.

Number of Incorrect Answers Number of Answer Options Credit Given (%)
0 2 100
1 2 0
2 2 0
Number of Incorrect Answers Number of Answer Options Credit Given (%)
0 3 100
1 3 0
2 3 0
3 3 0
Number of Incorrect Answers Number of Answer Options Credit Given (%)
0 4 100
1 4 50
2 4 0
3 4 0
4 4 0
Number of Incorrect Answers Number of Answer Options Credit Given (%)
0 5 100
1 5 50
2 5 25
3 5 0
4 5 0
5 5 0
Number of Incorrect Answers Number of Answer Options Credit Given (%)
0 7 100
1 7 50
2 7 25
3 7 0
4 7 0
5 7 0

8.5.6.2.1. Configure a Halves Checkbox Problem

To configure a halves checkbox problem, you add the partial_credit="halves" attribute to the <choiceresponse> element in the problem XML.

For example, the following XML shows the checkbox problem template updated to provide partial credit.

<choiceresponse partial_credit="halves">
  <checkboxgroup label="Which of the following is a fruit? Check all that apply.">
    <choice correct="true">apple</choice>
    <choice correct="true">pumpkin</choice>
    <choice correct="false">potato</choice>
    <choice correct="true">tomato</choice>
  </checkboxgroup>
</choiceresponse>

8.5.7. Checkbox Problem XML

8.5.7.1. Template

<problem>
<legend>Question text</legend>
<choiceresponse>
  <checkboxgroup label="label text">
    <choice correct="false">
      Answer option A (incorrect)
      <choicehint selected="true">
        Feedback for when learner selects this answer.
      </choicehint>
      <choicehint selected="false">
        Feedback for when learner does not select this answer.
      </choicehint>
    </choice>
    <choice correct="true">
      Answer option B (correct)
      <choicehint selected="true">
        Feedback for when learner selects this answer.
      </choicehint>
      <choicehint selected="false">
        Feedback for when learner does not select this answer.
      </choicehint>
    </choice>
    <choice correct="false">
      Answer option C (correct)
      <choicehint selected="true">
        Feedback for when learner selects this answer.
      </choicehint>
      <choicehint selected="false">
        Feedback for when learner does not select this answer.
      </choicehint>
    </choice>
    <compoundhint value="A B">
      Feedback when answers A and B are selected.
    </compoundhint>
   <compoundhint value="A C">
      Feedback when answers A and C are selected.
    </compoundhint>
  </checkboxgroup>
</choiceresponse>

<demandhint>
  <hint>Hint 1</hint>
  <hint>Hint 2</hint>
  <hint>Hint 3</hint>
</demandhint>

<solution>
  <div class="detailed-solution">
    <p>Solution or Explanation Heading</p>
    <p>Solution or explanation text</p>
   </div>
</solution>
</problem>

8.5.7.2. Tags

  • <choiceresponse> (required): Specifies that the problem contains options for learners to choose from.
  • <checkboxgroup> (required): Specifies that the problem is a checkbox problem.
  • <compoundhint> (optional): Specifies feedback for a specific combination of answers.
  • <choice> (required): Designates an answer option.
  • <demandhint> (optional): Specifies hints for the learner.

Tag: <choiceresponse>

Specifies that the problem contains options for learners to choose from.

Attributes

Attribute Description
partial_credit (optional) Specifies the type of partial credit given. EDC or halves.

Children

<checkboxgroup>

Tag: <checkboxgroup>

Specifies that the problem is a checkbox problem.

Attributes

Attribute Description
label (required) Specifies the name of the response field.

Children

<choice> <compoundhint>

Tag: <choice>

Designates an answer option.

Attributes

Attribute Description
true (at least one required) Indicates a correct answer. For checkbox problems, one or more <choice> tags can contain a correct answer.
false (at least one required) Indicates an incorrect answer.

Children

<choicehint>

Tag: <choicehint>

Specifies a hint for the answer option.

Attributes

Attribute Description
selected (required) true or false. Indicates if the hint is given when the answer option is selected, or when it is not selected.

Children

(none)

Tag: <compoundhint>

Designates feedback for a specific combination of answers.

Attributes

Attribute Description
value (at least one required) Indicates which combination of selected answers triggers this feedback.

Children

(none)

Tag: <demandhint>

Specifies hints available to the learner.

Children

<hint>

Tag: <hint>

Specifies a hint available to the learner.

Children

(none)